The Impacts of Upward 
Bound Math-Science on 
Postsecondary Outcomes 
7-9 Years After Scheduled 
High School Graduation 

Final Report 



Prepared by 

Neil S. Seftor 
Juan Carlos Calcagno 

Mathematica Policy Research, Inc. 



For 

U.S. Department of Education 
Office of Planning, Evaluation and Policy Development 
Policy and Program Studies Service 



2010 




This report was prepared for the U.S. Department of Education under Contract Number EA-97-0300001 with 
Mathematica Policy Research, Inc. The project monitor was Margaret Cahalan in the Policy and Program Studies 
Service. The views expressed herein do not necessarily represent the positions or policies of the Department of 
Education. No official endorsement by the U.S. Department of Education is intended or should be inferred. 

U.S. Department of Education 

Ame Duncan 
Secretary 

Office of the Planning, Evaluation and Policy Development 

Carmel Martin 
Assistant Secretary 

Policy and Program Studies Service 

Alan Ginsburg 
Director 

2010 

This report is in the public domain. Although permission to reprint this publication is not necessary, the suggested 
citation is: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and 
Program Studies Service, The Impacts of Upward Bound Math-Science on Postsecondary Outcomes 7-9 Years After 
Scheduled High School Graduation, Washington, D.C., 2010. 



To order copies of this report: 

Write to ED Pubs, Education Publications Center, U.S. Department of Education, NTIS, P.O. Box 22207, 
Alexandria, VA 22304. 

Or fax your request to 703-605-6794. 

Or e-mail your request to edpubs@inet.ed.gov. 

Or call in your request toll-free to 1-877-433-7827 (1-877-4-ED-PUBS). Those who use a telecommunications 
device for the deaf (TDD) or a teletypewriter (TTY) should call 1-877-576-7734. If 877 service is not yet available 
in your area, call 1-800-872-5327 (1-800-USA-LEARN). 

Or order online at http://edpubs.ed.gov. 

This report is also available on the Department’s Web site at 
http://www.ed.gov/about/offices/list/opepd/ppss/index.html. 

On request, this publication is available in alternate formats such as Braille, large print, or computer diskette. Eor 
more information, please contact the Department’s Alternate Eormat Center at 202-260-0852 or 202-260-0818. 




ACKNOWLEDGMENTS 



This report on the effects of Upward Bound Math-Science on students’ postsecondary 
outcomes reflects the contributions of many individuals. The authors would like to thank David 
Goodwin of the Policy and Program Studies Service at the U.S. Department of Education for his 
continued support, substantive guidance, and encouragement throughout the study. The authors 
would also like to thank Maggie Cahalan for her very valuable comments and other guidance. 

David Myers, who directed the national evaluation of Upward Bound for many years, 
played a key role in shaping the study design, led previous impact analyses. Mary Moore, a 
principal investigator for the national evaluation, was also instrumental in shaping the study 
design. Allen Schirm and Peter Schochet provided insightful suggestions throughout the analysis 
and drafts of the report. 

The authors would also like to thank the individuals who contributed to the data collection 
and analysis as well as to the production of the report. David DesRoches oversaw the data 
collection, and Zhanyun Zhao created the survey weights. Karin Zeller constructed the analysis 
variables and computed the impact estimates, and Mary Grider provided technical assistance 
with data and programming issues. Jennifer Baskwell produced the document. 




CONTENTS 



Chapter Page 

LIST OF EXHIBITS vii 

EXECUTIVE SUMMARY x 

I INTRODUCTION I 

A. UPWARD BOUND MATH-SCIENCE PROGRAM 2 

B. EVAEUATION OP THE UPWARD BOUND MATH-SCIENCE 

PROGRAM 3 

II THE OPERATION OP THE UPWARD BOUND MATH-SCIENCE PROGRAM 5 

A. PROJECT HOSTS AND STATE 6 

1. Host Institutions: Two- and Pour- Year Colleges and Universities 6 

2. Summer Program Staff and Project Director 6 

B . EEIGIBIEITY, RECRUITMENT AND ENROEEMENT, AND 

STUDENT CHARACTERISTICS 8 

1. Eligibility 8 

2 . Recruitment and Enrollment 10 

3 . Student Characteristics 12 

C. PROGRAM DESCRIPTION 14 

1. Setting 15 

2. Goals of the Program 15 

3 . Academic Orientation 16 

4 . Summer Offerings 17 

5. Academic- Year Offerings 20 

6. Instructional Approaches 22 

7. Intensity and Quantity of Services 24 

III THE IMPACTS OP THE UPWARD BOUND MATH-SCIENCE PROGRAM 26 

A. STUDY DESIGN 26 

1. Selecting the Samples 27 

2. Collecting Baseline Data 29 

3. Selecting a Matched Comparison Sample for the Impact Analysis 31 



V 




CONTENTS (continued) 

Chapter Page 

4. Estimating the Impacts of UBMS Participation 34 

B. DATA SOURCES AND OUTCOMES MEASURES 35 

1. Outcome Measures 35 

2. Data Sources 36 

3. Construction of the Outcome Measures 37 

C. ANAEYSIS AND EINDINGS 37 

1. The Effect of UBMS on Postsecondary Attendance, Persistence and 

Completion 38 

2. The Effect of UBMS on Postsecondary Eield of Study 39 

IV INTERPRETATION OE EINDINGS 49 

REEERENCES 52 

APPENDIX A: DATA COEEECTION AND OUTCOMES MEASURES A-1 

APPENDIX B: PROGRAM IMPACTS BY SUBGROUPS B-1 

APPENDIX C: SAMPEE SIZES AND STANDARD ERRORS C-1 



vi 




EXHIBITS 



Exhibit Page 

II. 1 Types of Institutions That Hosted Upward Bound Math-Science Projects, 1995 7 

11.2 Predominant Racial and Ethnic Group for Upward Bound Math-Science Staff, 

Summer 1998 9 

11.3 Racial and Ethnic Distribution for Upward Bound Math-Science Staff and Project 

Directors, Summer 1998 10 

11.4 Targets for Recruiting by Upward Bound Math-Science Projects, 1994 1 1 

11.5 Geographic Scope of Recruiting by Upward Bound Math-Science Projects, 1994 12 

11.6 Predominant Racial and Ethnic Group for Participants at Upward Bound Math- 

Science Projects, Summer 1998 14 

11.7 Most Important Goals of Upward Bound Math-Science Projects, Summer 1994 16 

11.8 Academic Orientation of Upward Bound Math-Science Projects 17 

11.9 Instruction Offered by Upward Bound Math-Science Projects, by Subject Area 19 

11.10 Noninstructional Services Offered by Upward Bound Math-Science Projects 20 

II. 1 1 Academic Year Services Offered by Upward Bound Math-Science Projects, 

1994-95 21 

11.12 Instructional Methods Used by Upward Bound Math-Science Projects, 1994 23 

11.13 Per-Capita Bunding for Upward Bound Math-Science Projects, EY 2004 25 

III . 1 B aseline Variables 31 

111.2 Summary Statistics Erom the Baseline Variables 33 

111. 3 Impact of Upward Bound Math-Science on Any Postsecondary Enrollment, 

Percentage of Students 41 

111.4 Impact of Upward Bound Math-Science on Highest Bevel of Postsecondary 

Enrollment: Eour-Year Institution, Percentage of Students 42 

111. 5 Impact of Upward Bound Math-Science on Highest Bevel of Postsecondary 

Enrollment: Two-Year Institution, Percentage of Students 43 

vii 




EXHIBITS (continued) 

Exhibit Page 

111.6 Impact of Upward Bound Math-Science on Highest Level of Postsecondary 

Enrollment: Other Institution, Percentage of Students 44 

111.7 Impact of Upward Bound Math-Science on Attendance at a Highly-Selective 

Four- Year Postsecondary Institution, Percentage of Students 45 

111.8 Impact of Upward Bound Math-Science on Total Postsecondary Credits Earned, 

Percentage of Students 46 

111.9 Impact of Upward Bound Math-Science on Completion of Any Credential and 

Highest Credential Completed (Intent To Treat), Percentage of Students 47 

III. 10 Impact of Upward Bound Math-Science on Postsecondary Field of Study (Intent 

To Treat), Percentage of Students 48 

B. 1 Impact of Upward Bound Math-Science on Any Postsecondary Enrollment by 

Prior Participation in Regular Upward Bound (ITT) B-2 

B.2 Impact of Upward Bound Math-Science on Highest Eevel of Postsecondary 
Enrollment by Prior Participation in Regular Upward Bound: Four- Year 
Institution (ITT) B-3 

B.3 Impact of Upward Bound Math-Science on Highest Eevel of Postsecondary 
Enrollment by Prior Participation in Regular Upward Bound: Two-Year 
Institution (ITT) B-4 

B.4 Impact of Upward Bound Math-Science on Highest Eevel of Postsecondary 

Enrollment by Prior Participation in Regular Upward Bound: Other Institution 
(ITT) B-5 

B.5 Impact of Upward Bound Math-Science on Attendance at a Highly Selective 
Four- Year Postsecondary Institution by Prior Participation in Regular Upward 
Bound (ITT) B-6 

B.6 Impact of Upward Bound Math-Science on Total Postsecondary Credits Earned 

by Prior Participation in Regular Upward Bound (ITT) B-7 

B.7 Impact of Upward Bound Math-Science on Completion of Any Credential and 
Highest Credential Completed by Prior Participation in Regular Upward Bound 
(ITT) B-8 

B.8 Impact of Upward Bound Math-Science on Postsecondary Field of Study by Prior 

Participation in Regular Upward Bound (ITT) B-9 

viii 




EXHIBITS (continued) 

Exhibit Page 

C. 1 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.3 C-2 

C.2 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.4 C-3 

C.3 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.5 C-4 

C.4 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.6 C-5 

C.5 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.7 C-6 

C.6 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.8 C-7 

C.7 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.9 C-8 

C.8 Sample Sizes and Standard Errors for Reported Impact Estimates: Exhibit III.IO ....C-9 

C.9 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B.l C-10 

C. 10 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B.2 C-11 

C. 1 1 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B.3 C-12 

C. 12 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B.4 C-13 

C. 13 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B.5 C-14 

C. 14 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B.6 C-15 

C. 15 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B. 7 C-16 

C. 16 Sample Sizes and Standard Errors for Reported Impact Estimates in Appendix B, 

Exhibit B. 8 C-17 



IX 




EXECUTIVE SUMMARY 



This final report updates the report Upward Bound Math-Science: Program Description and 
Interim Impact Estimates published in 2007 (Olsen et al. 2007). The 2007 interim report 
contained descriptive findings from a survey of Upward Bound Math-Science (UBMS) grantees 
from the late 1990s at the time of the study’s initiation and impact estimates through the period 
four to six years after expected high school graduation of the study sample. The current report 
presents impact estimates for the period seven to nine years after scheduled high school 
graduation. For context purposes we include descriptive information from the initial reports that 
gives a picture of the UBMS program as it was operating shortly after the time when the study 
sample members were participating in UBMS (1993-1995). It should be noted that the study 
sample and results represent the UBMS program as it was operating in the early years of its 
initiation. 

For many years, policy-makers have been concerned by the relatively low levels of 
academic achievement by economically disadvantaged K-12 students in math and science, by 
the underrepresentation of disadvantaged college students in math and science majors, and by the 
underrepresentation of people from disadvantaged groups in math and science careers. Although 
racial gaps in math and science test scores narrowed somewhat in the 1970s and 1980s, 
substantial gaps persisted through the 1990s to the present (U.S. Department of Education 
1996a). 

To help address these disparities, the U.S. Department of Education (ED) established a math 
and science initiative in 1990 within Upward Bound, a federal grant program designed to provide 
disadvantaged high school students with skills and experiences that will prepare them for college 
success. The initiative, referred to as Upward Bound Math-Science (UBMS), awards grants to 
institutions — largely colleges and universities — to operate UBMS projects. These projects were 
designed to differ from “regular” Upward Bound projects in several respects. To ensure that 
participants receive an intensive math and science precollege experience, UBMS projects 
provide instruction that includes hands-on experience in laboratories and computer facilities and 
at field sites. Opportunities are also provided to learn from mathematicians and scientists who 
are employed at the host institution or engaged in research or applied science in other institutions 
in the community. In addition to year-round services, a six-week summer program providing 
intensive instruction in laboratory science as well as mathematics through pre-calculus is also 
offered. 

Initially, ED funded 30 UBMS projects. By EY 2006, there were 126 UBMS projects 
serving 6,707 students at a total cost of $34 million. Therefore, the annual cost per student was 
approximately $4,990 in EY06. More than 80 percent of UBMS projects are hosted by four-year 
colleges and universities; most of the rest are hosted by two-year colleges (Curtin and Cahalan 
2004). 



X 




Participants in UBMS must meet the same eligibility requirements as regular Upward Bound 
students: students must (a) belong to families classified as low-income (taxable income of no 
greater than 150 percent of the poverty line) or (b) be a potential first-generation college student 
(neither parent has a bachelor’s degree) or (c) have a need for academic support, as determined 
by the grantee. Some students who participate in UBMS summer programs are referred from 
regular Upward Bound programs and then return to those programs during the academic year, 
but the vast majority of UBMS students do not participate at all in regular Upward Bound. 
However, as would be expected, UBMS projects are more likely to consider students’ interests in 
math and science when reviewing applications than are most regular Upward Bound projects 
(Moore 1997). While 25 percent of participants are white, most program participants are from 
underrepresented minority groups: about 60 percent of participants are African American or 
Hispanic (Curtin and Cahalan 2004). 



EVALUATION OF UPWARD BOUND MATH- SCIENCE 

Since 1991, Mathematica Policy Research, Inc. (Mathematica) has been conducting the 
National Evaluation of Upward Bound for ED. The centerpiece of this evaluation has been a 
random assignment evaluation of regular Upward Bound. In 1997, ED added a new component 
to the evaluation that is focused on UBMS. In 1998, Mathematica selected a random sample of 
the students who participated in UBMS between 1993 and 1995 at projects that were still 
operating at that time. This report constitutes the second of two evaluation reports on UBMS, 
and it is based on participant surveys and student transcripts collected for this sample between 
2003 and 2004. The first report was based on data collected between 2001 and 2002, focusing on 
high school outcomes and preliminary postsecondary outcomes. 

The evaluation of UBMS has two components: a descriptive analysis and an impact 
analysis. The descriptive analysis relies primarily on a survey of project directors to describe the 
resources available to UBMS projects; the types of institutions that host them; the credentials and 
demographic characteristics of project staff members; recruitment, eligibility and enrollment of 
students; student characteristics; and program offerings. The impact analysis is designed to 
measure the effects of UBMS on (a) performance in high school, especially in math and science 
courses; (b) postsecondary attendance, persistence and completion; and (c) the likelihood of 
completing a postsecondary degree in mathematics or a scientific field. The interim report (Olsen 
et al. 2007) presented detailed findings on the effects of UBMS on high school outcomes. By the 
1999-2000 data collection on which that report was based, nearly all sample members had 
progressed out of high school and into college. Therefore, this report focuses only on 
postsecondary outcomes. 

The impact analysis is based on a comparison of UBMS participants with a sample of 
students that (a) applied to enroll in regular Upward Bound programs in the early 1990s, (b) 
never participated in UBMS, and (c) have been tracked by Mathematica as part of the national 
evaluation. This comparison group was selected using propensity modeling to ensure that it had 
similar characteristics to the sample of UBMS participants, and we controlled statistically for the 
small remaining differences in these characteristics between UBMS participants and the 
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comparison group. Because some of the matehed eomparison group sample partieipated in 
regular Upward Bound, the report presents separate impact estimates for those who participated 
in Upward Bound and those who did not participate. 

If UBMS participants are more interested or skilled in math and scienee than the students in 
the comparison group, the estimated effects of the program may be subject to “selection bias” 
and may overstate the true effects of participating in UBMS. Moore (1997) described UBMS 
participants as a more select group than regular Upward Bound partieipants based on UBMS 
students having earned somewhat higher grades and having greater interest in math and seienee 
(23, 26) before participating in Upward Bound. Discussions with UBMS and regular Upward 
Bound staff members revealed that UBMS participants were typically considered “more serious 
about sehool” than regular Upward Bound participants (Moore 1997, 26). 

However, we believe that the comparison group we selected was the best choice available 
because random assignment was not possible. Our analysis shows that the participant and 
comparison groups are similar in other ways such as academic course-taking and aehievement 
before program participation. In addition, we implemented a data eolleetion and analysis plan 
designed to minimize selection bias (see Chapter III for more details). 

Note that the descriptive findings and impaet estimates presented in this report describe the 
operations and effects of the Upward Bound Math-Scienee Program as it operated in the mid- 
1990s. At that time, it was a relatively new program, and some changes have occurred since that 
time in how UBMS projects operate. In Chapter II, we mention some of these changes as they 
are refleeted in information provided to us by UBMS projeet direetors in a survey of grantees. It 
is certainly possible that some of the ehanges in the program since the mid-1990s have 
influenced the effectiveness of UBMS projects, and the evaluation does not attempt to measure 
any ehanges in effeetiveness sinee that time. In this report, we measure the effeets of the program 
on people who partieipated between 1993 and 1995 and describe the operations of the program at 
that time. 



REPORT FINDINGS 

Given the aeademic serviees provided by UBMS, it is natural to ask whether partieipating in 
UBMS affeets the educational outcomes of the students who partieipate. From our impact 
analysis, we found that the UBMS program achieved the following: 



Increased enrollment at four-year institutions. The impact estimates were all 
positive (ranging from 8.6 to 18.2 pereentage points, with an average of 12.0) and 
signifieant for attending a four-year institution. 




• Shifted enrollment from two-year to four-year institutions for Upward Bound 
participants. Almost all impacts on attending a four-year institution are larger for 
UMBS participants without previous participation in regular Upward Bound. 

Moreover, the negative effect on enrollment in a two-year college is driven almost 
exclusively by the impact for students who did not participate in regular Upward 
Bound. Combined, these findings suggest that UBMS increases four-year enrollment 
for both regular Upward Bound participants and nonparticipants, but leads to an 
additional shift from two-year to four-year enrollment for those who did not have 
prior regular Upward Bound participation. 

• Increased enrollment at more selective institutions. A substantial portion of UBMS 
participants who attended four-year institutions attended more- selective institutions, 
with positive (ranging from 12.2 to 22.9 percentage points, with an average of 18.6) 
and statistically significant differences for all of the measures. 

• Increased math and science course taking. Consistent with the primary objective of 
UBMS, the program increased the number of postsecondary credits earned in math 
and science from 21.6 to 29.5. 

• Increased postsecondary degree completion overall and at four-year institutions. 
UBMS increased the likelihood of completing any postsecondary credential (ranging 
from 9.0 to 12.1 percentage points, with an average of 11.0) (Exhibit III.9). This 
increase in overall postsecondary completion is primarily attributable to an increase 
in the percentage of sample members whose highest credential was a bachelor’s 
degree (ranging from 11.8 to 16.8 percentage points, with an average of 15.7, and 
statistically significant for all of the measures). 

• Increased likelihood of earning a degree in a social science field of study. UBMS 
students showed statistically significant effects on the likelihood of earning a degree 
or certificate in the social sciences field and in earning a degree in a social science 
field from a four-year college or university. UBMS students showed positive effects 
in the direction of increasing the likelihood of majoring in math and physical science 
fields. However, the effects were not statistically significant. 

It is tempting to compare the estimated impacts of UBMS with the estimated impacts of 
regular Upward Bound presented in earlier reports. However, it is important to recognize that the 
two studies used different methods: although the evaluation of regular Upward Bound is based 
on an experimental design, the “gold standard” in evaluation research, the evaluation of UBMS 
is based on nonexperimental methods that may suffer from selection bias, as described earlier. If 
the estimated effects of UBMS are inflated because of selection bias, then the impression based 
on our findings that UBMS is more effective than regular Upward Bound might be attributable to 
differences in the methods used to estimate the impacts instead of differences in the effectiveness 
of the two programs. 




I. INTRODUCTION 



For many years, policy-makers have been concerned by the relatively low levels of 
academic achievement by economically disadvantaged K-12 students in math and science, by 
the underrepresentation of disadvantaged college students in math and science majors, and by the 
underrepresentation of people from disadvantaged groups in math and science careers. National 
statistics show that although the math and science test scores gaps between minority students and 
white students narrowed somewhat in the 1970s and 1980s, gaps in test scores and other 
educational outcomes persisted through the 1990s to the present:^ 



• When the Upward Bound initiative started, disadvantaged students took fewer 
math and science courses in high school. In the 1991-92 school year, 57 percent of 
seniors in the lowest socioeconomic status (SES) quartile took a math course 
compared with 75 percent of seniors from the highest SES quartile; 37 percent of 
seniors from the lowest SES quartile took a science course compared with 61 percent 
of seniors from the highest SES quartile (U.S. Department of Education 1996b). In 
1994, only 58 percent of black high school graduates had completed geometry while 
in high school compared with 73 percent of white high school graduates. In the same 
year, only 13 percent of black and Hispanic graduates had completed the common 
triad of science courses — biology, chemistry, and physics — compared with 23 
percent of white graduates (U.S. Department of Education 1996a). 

• Minority college students were less likely to take math and science courses or earn 
a degree in math or science. Ten percent of black college students and 14 percent of 
Hispanic college students received credit for calculus or advanced math courses in 
the late 1980s compared with 22 percent of white college students. Sixteen percent 
of black college students and 21 percent of Hispanic college students earned course 
credits in chemistry compared with 27 percent of white college students, and 8 
percent of black students and 1 1 percent of Hispanic students earned college credit 
for physics compared with 18 percent of white students (U.S. Department of 
Education 1994). Because minority students earned fewer college credits in math and 
science (biological sciences and life sciences, computer and information sciences, 
engineering, engineering-related technologies, mathematics, and physical sciences 
and science technologies) than white students, it is not surprising that they were less 
likely to earn degrees in those subjects. Black students earned 7 percent of all 
bachelor’s degrees in 1995-96, but just 7 percent of all bachelor’s degrees in math 



1 . Ideally, socioeconomic measures such as income would be used to define groups rather than race or ethnicity. For 
most education outcomes of interest, however, data are not presented on different income groups. Because racial and 
ethnic minorities are disproportionately lower income (U.S. Census Bureau 2001, 40), data based on race and 
ethnicity offer a reasonable, albeit imperfect, estimate of economically disadvantaged students’ educational 
experiences. 
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and science fields. In the same year, Hispanic students earned 5 percent of all 
bachelor’s degrees, but just 4 percent of all bachelor’s degrees in math and science 
(U.S. Department of Education 1999). 

• Minority students are underrepresented in math and science fields. For example, 
about 8 percent of bachelor’s degrees in 1998 were awarded to blacks, and about 7 
percent were awarded to Hispanics; these two groups made up about 12 percent and 
9 percent, respectively, of the United States population in the same period (National 
Science Foundation n.d.; Census Bureau 2001). 



A. UPWARD BOUND MATH-SCIENCE PROGRAM 

To help address these disparities, the U.S. Department of Education (ED) in 1990 
established the Upward Bound Math-Science Program (UBMS) within Upward Bound, a federal 
grant program designed to provide disadvantaged high school students with skills and 
experiences that would prepare them for college success. UBMS was designed to differ from 
“regular” Upward Bound in a few key respects. To ensure that participants receive an intensive 
math and science precollege experience, ED requires UBMS projects to provide instruction that 
includes hands-on experience in laboratories and computer facilities as well as at field sites. Also 
provided are the following: opportunities to learn from mathematicians and scientists employed 
at the host institution or engaged in research or applied science in other institutions in the 
community; involvement with tutors and counselors who are graduate and undergraduate math 
and science majors; and a six- week summer program consisting of daily course work and 
activities, instruction in laboratory science as well as mathematics through pre-calculus (in 
addition to foreign language, composition and literature, which are also required offerings at 
regular Upward Bound projects). 

Initially, ED funded 30 UBMS projects. By FY 2006, there were 125 UBMS projects 
serving 6,707 students at a total cost of $32.3 million. Therefore, the annual cost per student — 
approximately $4,800 — is comparable in cost with regular Upward Bound, but much more 
expensive than other federally funded precollege programs. More than 80 percent of UBMS 
projects are hosted by four-year colleges and universities; most of the rest are hosted by two-year 
colleges (Curtin and Cahalan 2004). 

UBMS participants must meet the same eligibility requirements as regular Upward Bound 
participants: they must (a) come from families that are classified as low income (taxable income 
not more than 150 percent of the poverty line) or (b) be a potential first- generation college 
student (neither parent has a bachelor’s degree). Some students who participate in UBMS are 
referred from regular Upward Bound programs and then return to those programs during the 
academic year, but the vast majority of UBMS students do not participate at all in regular 
Upward Bound. However, as would be expected, UBMS projects are more likely to consider 
students’ interests in math and science when reviewing applications than are most regular 
Upward Bound projects (Moore 1997). While 25 percent of participants are white, most program 
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participants are from underrepresented minority groups: about 60 percent of participants are 
African American or Hispanic (Curtin and Cahalan 2004). 

Despite coming from low-income families, the evidence suggests that on average, UBMS 
serves students do well in high school and attend college at higher rates than the average low- 
income student. Data reported by Upward Bound projects suggest that before participating in 
Upward Bound, UBMS participants earned higher grades on average than regular Upward 
Bound participants (Curtin and Cahalan 2004). In addition, the national evaluation has shown 
that regular Upward Bound participants would have attended college at much higher rates than 
the average low-income student even if they had not participated in Upward Bound (Seftor, 
Mamun, and Schirm 2009). Nationwide, among all students in eighth grade in 1988, 
approximately 76 percent enrolled in postsecondary education within about eight years after high 
school (Ingels et al. 2002). Among disadvantaged students, the national postsecondary 
enrollment rate was much lower — less than 60 percent for students who were in the lowest SES 
quartile or whose parents did not attend college. In comparison, 81 percent of Upward Bound 
applicants assigned to the control group enrolled in postsecondary education within seven to nine 
years after high school. Therefore, the evidence strongly suggests that UBMS serves high school 
students who are much more likely to attend college than the average low-income student. 

B. EVALUATION OF THE UPWARD BOUND MATH-SCIENCE PROGRAM 

The legislation establishing Upward Bound authorizes ED to sponsor studies of it, including 
examinations of program effectiveness. In 1991, ED awarded a contract to Mathematica Policy 
Research, Inc. (Mathematica) to conduct the national evaluation of Upward Bound. This 
evaluation has several components, but its signature feature is an experiment to measure the 
effects of participating in regular Upward Bound. We selected a random sample of Upward 
Bound projects (excluding UBMS projects). Eor each of these projects, we randomly assigned 
eligible applicants to a treatment group, which was offered the chance to participate in the 
program, or a control group, which was not. The evaluation, which only recently ended, was one 
of the first to use experimental methods to measure the effects of a federally funded education 
program. 

This report presents the results of an evaluation of the Upward Bound Math-Science 
Program. In 1997, Mathematica completed two reports on UBMS. One provided a descriptive 
analysis of the program based primarily on site visits to a representative sample of 14 UBMS 
projects (Moore 1997). The other provided an assessment of the feasibility of conducting a 
rigorous evaluation of the effects of UBMS on student outcomes (Myers 1997). When ED 
awarded a contract to Mathematica in 1997 to extend its evaluation of the effects of regular 
Upward Bound, it also specified an evaluation of the effects of UBMS. This evaluation consists 
of two components: a descriptive analysis and an impact analysis. The descriptive analysis relies 
primarily on a survey of UBMS project directors conducted in the spring of 1998. The analysis is 
designed to describe the resources available to UBMS projects; the types of institutions that host 
them; the credentials and demographic characteristics of project staff members; recruitment, 
eligibility and enrollment of students; student characteristics; and a description of the program, 
including its goals, academic orientation and instructional methods as well as the intensity and 
quantity of the services provided. 
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The UBMS impact study is designed to measure the effects of participating in UBMS on 
college enrollment, choice of major, and other outcomes for students who participated during the 
summer of 1993, 1994 or 1995. Because the sample was not selected until 1998, we restricted 
the sample to participants at UBMS projects that were still operating that year: obtaining lists of 
participants from programs that were no longer operating in 1998 would have been nearly 
impossible. Conceptually, the study contrasts how participants fared with how they would have 
fared if they had not participated in UBMS. We compared UBMS participants with eligible 
applicants to the regular Upward Bound projects participating in the national evaluation. From 
this pool, we systematically selected a matched comparison group of students who were as 
similar as possible to UBMS participants in terms of characteristics and experiences that could 
potentially predict later outcomes. These characteristics included demographics such as sex, race 
and ethnicity as well as prior academic achievement such as grade point average (GPA) and 
math and science courses taken in ninth grade. The key difference was that the matched 
comparison students did not participate in UBMS. 

The selection of matched comparison students also took into account experiences in other 
precollege programs, in particular, regular Upward Bound. Because regular Upward Bound is an 
intensive program that can influence high school achievement and postsecondary outcomes 
(Myers and Schirm 1999; Myers et al. 2004; Seftor, Mamun, and Schirm 2009), it is important to 
account for exposure to regular Upward Bound when estimating how UBMS participants would 
have fared if they had not participated in UBMS. For UBMS participants who had previously 
participated in a regular Upward Bound program — perhaps during the academic year — we 
selected comparison students who had also participated in the regular Upward Bound; these 
comparison students were selected from the treatment group for the evaluation of regular 
Upward Bound. For UBMS participants who had not participated in regular Upward Bound, we 
selected comparison students who did not participate in regular Upward Bound; these 
comparison students were selected from the control group for the evaluation of regular Upward 
Bound. 

The treatment and comparison groups were enrolled in high school at the same period of 
time. However, because of the later addition of the UBMS analysis to the original Upward 
Bound evaluation, some pieces of data were collected at different points in time. As a result, 
several data sources play a key role in the impact analysis. Baseline characteristics were 
collected for comparison group members through the baseline survey for the evaluation of 
regular Upward Bound. Baseline information on many of the same characteristics was collected 
for UBMS participants through a follow-up survey conducted in 1999. Although the 1999 survey 
was conducted four to six years after our sample had participated in the program, most of the 
baseline information collected — including sex, race and ethnicity — is time invariant. Follow-up 
surveys of treatment and comparison group members were also conducted in 2001-2002 and 
2003-2004 to collect information about educational outcomes. Secondary and postsecondary 
transcripts were collected for both types of students to assess academic achievement. 
Additionally, data were collected from two administrative data sources: the National Student 
Clearinghouse (NSC) and the federal Student Financial Aid (SFA) records. 
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At the time of the follow-up survey in 2003-2004, the majority of sample members had 
been out of high school for about seven to nine years, and more than half had received 
postsecondary credentials. This report updates previous estimates of the effect of UBMS on 
postsecondary outcomes, including enrollment, persistence and completion. Chapter II of this 
report describes the operation of the UBMS program, followed by the findings from the impact 
analysis in Chapter III. Appendices A-C provide additional information on the study and study 
findings. Appendix A presents a focused look at data sources and outcomes measures; Appendix 
B provides tables detailing program impacts by subgroups; and Appendix C presents tables 
describing sample sizes and standard errors. 



II. THE OPERATION OF THE UPWARD BOUND MATH-SCIENCE PROGRAM 

It is necessary to understand what the UBMS program entails to interpret information on its 
impacts. This chapter describes key features of the operations of UBMS projects, including the 
characteristics of host institutions and staff members, projects’ recruitment practices and 
enrollment levels, participants’ characteristics, and projects’ goals and services. For context, this 
chapter presents comparable information on the operations of regular Upward Bound when 
possible. 

The primary data source for this chapter is a survey of UBMS projects conducted in spring 
1998. The survey sample consisted of all 81 projects operating at the time, and 74 of the 81 
projects responded to the survey. The survey requested information about program operations in 
two separate years — 1994 (in the middle of the period over which our sample was participating 
in UBMS) and 1998 (the year before the survey) — but some questions were specific to 1998. 
When possible, we focus our analysis of program operations on 1994 to facilitate comparisons 
with regular Upward Bound projects operating in 1993, as reported in Fasciano and Jacobson 
(1997), and to describe the programs that served the same cohorts of participants for whom we 
measured the impacts of the program (see Chapter III); unless noted otherwise, the results for 
1998 were generally similar to those for 1994. To augment the information provided by the 
survey of UBMS projects, we also use information from case studies and annual performance 
reports (Moore 1997). 

The findings in this chapter indicate that UBMS projects provide intensive academic 
enrichment to disadvantaged high school students in math and science by using staff members 
with strong academic credentials in those subjects. Some of the features that make UBMS 
projects distinctive, even from regular Upward Bound projects, are (a) high levels of annual 
funding per student and low student-teacher ratios, (b) recruiting strategies that attract students 



2. We did not adjust (weight) for survey nonresponse, reasoning that the number of nonrespondents was low enough 
to eliminate any serious concerns about whether the data represented the sample. Also, rarely did more than three 
UBMS projects fail to respond to any particular item on the questionnaire. We excluded the one project that reported 
serving only veterans in 1998. Note that veterans’ projects were also excluded from the survey of regular Upward 
Bound grantees, so the comparisons that are made in this chapter between the two types of Upward Bound programs 
are based on Upward Bound projects that did not exclusively serve veterans. 
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from wide geographic areas, (c) the providing of services that are heavily concentrated in 
residential programs during the summer, (d) course offerings that focus on math and science 
relative to other subjects, (e) academic preparation over nonacademic college preparatory 
activities, and (f) academic enrichment over remediation. The remainder of the chapter provides 
a description of UBMS and an assessment of its distinctive features. 



A. PROJECT HOSTS AND STAFF 

The impacts of UBMS projects on student outcomes may depend on the types of institutions 
that host them and the people they hire to serve as instructors and other staff members. In this 
section, we describe the types of institutions that host UBMS projects and the staff members who 
provide services to program participants. 



1. Host Institutions: Two- and Four- Year Colleges and Universities 

The types of institutions that host a UBMS project may influence where students attend 
college. Most Upward Bound programs are hosted by either two- or four-year postsecondary 
institutions. Evidence from the national evaluation of regular Upward Bound suggests that 
participation at projects hosted by four-year colleges raises the probability of attending a four- 
year college, and participation at projects hosted by two-year colleges raises the probability of 
attending a two-year college (Myers, Olsen, and Seftor 2002). Therefore, the types of institutions 
that host UBMS projects may influence the types of postsecondary institutions that program 
participants subsequently attend. 

Nearly nine out of ten UBMS projects operating in the mid-1990s were hosted by four-year 
colleges, a substantially higher proportion than among regular Upward Bound projects (see 
Exhibit II. 1). Eour-year colleges may find it easier than other potential host institutions to meet 
some of ED’s guidelines for UBMS, including offering hands-on experience in laboratories and 
computer facilities, providing opportunities to learn from mathematicians and scientists engaged 
in research or applied science, and supporting involvement with tutors and counselors who are 
graduate and undergraduate students in math and science. 



2. Summer Program Staff and Project Director 

UBMS projects are directed by highly educated individuals and are staffed by people with 
strong credentials in math and science. These staff members have responsibility for a relatively 
small number of students, which may provide opportunities for individual instruction. At the 
typical project, the project director and staff members can provide same-race role models for 
many of the students they serve. The sections below provide more detail on our findings 
concerning staff size and composition by job title, staff credentials, and the racial composition of 
the project staff. 
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EXHIBIT II. 1 



TYPES OE INSTITUTIONS THAT HOSTED UPWARD BOUND MATH-SCIENCE PROJECTS, 1995 



Type of Institution 


Upward Bound Math-Science (%) 


Regular Upward Bound (%) 


Pour-year college or university 


88 


68 


Two-year college 


II 


28 


Other institution 


2 


4 



Source; Reprinted from Mary T. Moore, Developing Math and Science Skills Among Disadvantaged Youth: A 
Review of the Upward Bound Precollege Math/Science Centers (Washington, D.C.: Department of Education, 
Planning and Evaluation Service, September 1997), Exhibit II. I, 15. 



Note: Percentages may not total 100 percent due to rounding. 



a. Staff Size and Composition by Job Title 

In 1994, UBMS projects had an average of 24 staff members, comprising roughly eight 
instructors, five resident counselors, four mentors, three tutors, two administrators, one academic 
or guidance counselor, and one clerical staff member; by 1998, the average Math and Science 
Center (MSC) had almost 26 staff members, including 9 instructors. Overall, the average 
student- staff ratio in summer 1998 was 2:1, with a range from about 1:1 to 5:1. 

These findings, combined with findings from Moore (1997), suggest that student-staff ratios 
are typically lower in UBMS projects than in regular Upward Bound projects. The survey of 
grantees did not collect information on the number of full-time equivalent (FTE) staff members, 
but the information available suggests that UBMS projects typically maintain student-staff ratios 
that are substantially lower than in regular Upward Bound. Moore (1997) found that 14 
randomly selected UBMS projects visited in summer 1996 had an average of 2.6 students per 
FTE staff member, including administrators, and 8.2 students per FTE instructional staff 
member. In contrast, tabulations from the data used by Fasciano and Jacobson (1997) indicate 
that in summer 1992, regular Upward Bound projects had more students per staff member — 5.1 
students per FTE staff member and 13.6 students per FTE instructional staff member (see 
Exhibit II. 6 in U.S. Department of Education, Office of the Under Secretary 1997). 



b. Credentials 

At the average UBMS project in 1998, most staff members were highly educated and had 
educational backgrounds in math and science. About one-quarter had attended some college but 
had not obtained a degree, another quarter had obtained a bachelor’s or associate’s degree 
(mostly bachelor’s degrees), and the rest had done graduate work or had obtained a graduate 
degree. Most staff members without undergraduate degrees were undergraduate students who 
served as UBMS mentors, tutors and resident counselors while working toward bachelor’s 
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degrees in math, science or education. Approximately two out of five staff members had their 
highest degree in science or the social sciences (31 percent) or math (10 percent). 

Most instructors at the average UBMS project had experience teaching math or science. 
During the school year, most instructors were either high school teachers (41 percent) or 
postsecondary teachers (31 percent); one-fifth were graduate students (14 percent) or 
undergraduates (6 percent). Moreover, at the typical project, two-thirds of the high school 
teachers and three-quarters of the postsecondary teachers taught in a math or science field. 

The professional and educational backgrounds of UBMS project directors provide insight on 
their credentials to direct projects. One-fifth of the directors were faculty members at the host 
institution or another college, roughly the same percentage as in regular Upward Bound. In 1998, 
two-thirds of the directors held a master’s degree and one-fifth held a doctorate. About half had 
their highest degree in education, less than one-fifth had their highest degree in engineering, 
mathematics, or physical sciences, and 12 percent had their highest degree in the social sciences. 
UBMS project directors were less likely than the program staff as a whole to have a background 
in math or science. Compared with regular Upward Bound project directors, UBMS project 
directors were twice as likely to have a doctorate. 



c. Race and Ethnicity 

At about 9 out of 10 UBMS projects operating in 1998, one racial or ethnic group accounted 
for a more than half of the staff members (see Exhibit II. 2). In many cases this pattern may have 
reflected a conscious strategy, also used in regular Upward Bound, to provide minority students 
with same -race role models. For example, at 21 UBMS projects, a majority of the staff members 
were black; at 18 of those, a majority of the students were also black. 

The racial and ethnic profile of UBMS project directors was similar to that of the UBMS 
project staff (see Exhibit II. 3). Both staff members and project directors were nearly evenly split 
between white and nonwhite; project directors were slightly more likely than other staff 
members to be black and were less likely to be Asian than other staff members. The race and 
ethnicity of the UBMS project director often matched that of the predominant student racial and 
ethnic group: more than three-fourths of the minority directors headed programs where students 
from the same group constituted a plurality of participants. 



B. ELIGIBILITY, RECRUITMENT AND ENROLLMENT, AND STUDENT 
CHARACTERISTICS 

To shed light on the types of students who participate in UBMS, we examine the eligibility 
criteria that students must meet and the recruiting strategies that UBMS projects use to attract 
students. We also examine the characteristics of participants as reported by project directors. 
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1. Eligibility 

UBMS projects have to meet the same federal rules as regular Upward Bound projects 
concerning the composition of participants. At each project, at least two-thirds of the participants 
must be both low-income and potential first- generation college students; the remaining students 
must meet either of these two criteria. At the average project in summer 1998, about 77 percent 
of students met both of these eligibility criteria, about 14 percent were first-generation only, and 
about 9 percent were low-income only, very similar to the distribution in regular Upward Bound 
during 1992-93. In addition, UBMS and regular Upward Bound projects are allowed only to 
serve students who have completed eighth grade. 



EXHIBIT II.2 

PREDOMINANT RACIAL AND ETHNIC GROUP EOR UPWARD BOUND MATH-SCIENCE STALE, 

SUMMER 1998 



Majority white 
54% 




Source: 1998 survey of Upward Bound Math-Science projects. 

Note: Percentages may not total 100 percent due to rounding. The 12 percent of projects with no majority racial and 
ethnic group included 6 percent of projects with a plurality of black students and 4 percent with a plurality of 
white students. 
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EXHIBIT II.3 



RACIAL AND ETHNIC DISTRIBUTION EOR UPWARD BOUND MATH-SCIENCE STALE AND PROJECT 

DIRECTORS, SUMMER 1998 



Race and Ethnicity 


Staff (%) 


Director (%) 


White 


49 


49 


Black 


33 


39 


Hispanic or Latino 


7 


7 


Asian 


4 


I 


American Indian or Alaskan Native 


2 


4 



Source: 1998 survey of Upward Bound Math-Science projects. 

Note: Percentages may not total 100 percent because (a) people may fall into multiple categories, (b) the Pacific 
Islander category was excluded from the exhibit because only nine staff members nationwide fell into this 
category, and (c) some staff members may not have been classified by race or ethnicity. 



Most UBMS projects also adopt additional student eligibility criteria for enrollment in the 
program — for example, requirements for completion of additional grades beyond grade 8, school 
course work or recommendations. In 1994, more than three-fourths of projects required students 
to have finished ninth grade; a few projects required 10th grade completion. In addition, nearly 
all UBMS projects required participants to have a teacher recommendation and to have 
completed at least one high school course in math or science, and those applicants enrolled in 
regular Upward Bound commonly needed a recommendation from the director. Finally, about 30 
percent of UBMS projects in 1994 prohibited students from attending the project again after the 
previous summer’s program, and almost half prohibited students from attending the project again 
unless they met certain criteria. These practices changed dramatically by 1998 when only 7 
percent of UBMS projects prohibited any participating students from attending the project again 
after the prior summer, and 71 percent prohibited students from returning unless they met certain 
criteria. The questionnaire did not address the specific types of criteria that projects imposed. 



2. Recruitment and Enrollment 

To find a pool of potentially eligible applicants, UBMS projects focused mainly on other 
precollege programs or secondary schools (see Exhibit II.4). Among UBMS projects operating in 
summer 1994, nearly all recruited from regular Upward Bound projects while substantial 
majorities also recruited directly from Talent Search projects and from middle or high schools. 
However, it is important to note that although almost all UBMS projects made efforts to recruit 
from regular Upward Bound projects in 1994, data from the evaluation suggest that fewer than 
one in five UBMS participants had actually previously participated in regular Upward Bound. 

Historically, UBMS projects have cast a wide net in recruiting students beyond the local 
areas of the host institutions. In 1994, only about one in ten UBMS projects recruited exclusively 
from a specific and typically local set of feeder schools or Upward Bound projects; the rest 
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recruited from statewide or regional lists of schools and programs (see Exhibit II.5). By 1998, 
however, the percentage of UBMS projects that recruited exclusively from a specific set of 
feeder schools or Upward Bound projects had tripled. Therefore, it appears that, over time, more 
UBMS projects are using strategies that focus on recruiting from local projects like regular 
Upward Bound projects. 

By design, UBMS projects are smaller than regular Upward Bound projects. Through 
recruitment, UBMS projects received an average of 108 applications for the summer of 1994, 
ranging from a low of 50 to a high of 300, and they enrolled between 40 and 53 students. In 
contrast, regular Upward Bound programs enrolled an average of about 75 students in the mid- 
1990s (U.S. Department of Education, Office of the Under Secretary 1997). 



EXHIBIT II.4 

TARGETS EOR RECRUITING BY UPWARD BOUND MATH-SCIENCE PROJECTS, 1994 




Percent of Upward Bound Math-Science Projects 
Source: 1998 survey of Upward Bound Math-Science projects. 
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EXHIBIT II.5 



GEOGRAPHIC SCOPE OE RECRUITING BY UPWARD BOUND MATH-SCIENCE PROJECTS, 1994 




45% 



Source; 1998 survey of Upward Bound Math-Science projects. 
Note: Percentages may not total 100 percent due to rounding. 



3. Student Characteristics 

In our 1998 survey of UBMS projects, project directors provided information on the 
distribution of students participating in their projects by sex, race, grade level and place of 
residence: 



• Sex. At the average UBMS project, like the average regular Upward Bound project, 
60 percent of students were female (based on data for all UBMS projects operating 
in 1998 and all regular Upward Bound projects operating in 1992). However, this 
percentage varied considerably across UBMS projects, from a low of 25 percent to a 
high of 78 percent. 

• Race. On average, UBMS projects served an ethnically diverse group of students: 42 
percent black, 27 percent white, 15 percent Hispanic, 8 percent Asian, 5 percent 
American Indian, and 1 percent Native Hawaiian or other Pacific Islander. However, 
reflecting the demographics of the region served, many UBMS projects serve 
participants where one racial and ethnic group constituted more than half (see 
Exhibit II. 6). In comparison, at 87 percent of regular Upward Bound projects 
operating in 1992-93, one racial and ethnic group accounted for a majority of 
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participants. Furthermore, some UBMS projects exclusively served students from a 
single racial or ethnic group. For example, six UBMS projects reported that all of its 
participating students were black. 

• Grade level. The eligibility guidelines discussed above, along with other factors, can 
affect the distribution of students across different grade levels. At the average 
project, 29 percent of participants were entering 12th grade, 37 percent were 
entering 1 1th grade, 27 percent were entering 10th grade, and 6 percent were 
entering 9th grade. In regular Upward Bound during the summer of 1992, 20 percent 
of participants were entering 12th grade, 32 percent were entering 1 1th grade, 31 
percent were entering 10th grade, and 16 percent were entering 9th grade. These 
exhibits suggest that, on average, UBMS projects serve students who are slightly 
closer to graduation than is the case at regular Upward Bound projects. However, 
there was substantial variation in the grade-level distribution of participants across 
projects in 1998. For example, one UBMS project reported that all of its participants 
were entering 12th grade while four reported that none were rising seniors. In 
addition, the proportion of rising juniors ranged from 0 to 72 percent, and the 
proportion of rising sophomores ranged from 0 to 60 percent. 

• Place of residence. Given that most projects recruited across the state or region, it is 
not surprising that many UBMS participants came from outside the grantee’s local 
city or town. At the average project, only about 25 percent of the students were 
locals. As we would expect, projects that recruited only from a set of local schools 
or regular Upward Bound projects served considerably higher percentages of 
students from the local area than other UBMS projects. 
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EXHIBIT II.6 



PREDOMINANT RACIAL AND ETHNIC GROUP EOR PARTICIPANTS AT UPWARD BOUND MATH- 

SCIENCE PROJECTS, 

SUMMER 1998 




Source: 1998 survey of Upward Bound Math-Science projects. 

Note: Percentages may not total 100 percent due to rounding. The 37 percent of Upward Bound Math-Science 
projects with no majority racial and ethnic group of students included 17 percent with a plurality of white 
students, 1 1 percent with a plurality of Hispanic students, 6 percent with a plurality of black students, 3 percent 
with a plurality of American Indian students, and I percent with a plurality of Asian students. 



Moore (1997) described UBMS participants as a more select group than regular Upward 
Bound participants based on their having earned somewhat higher grades and having expressed 
greater interest in math and science (23, 26) before participating in Upward Bound. Discussions 
with UBMS and regular Upward Bound staff members revealed that UBMS participants were 
typically considered “more serious about school” than regular Upward Bound participants 
(Moore 1997, 26). 



C. PROGRAM DESCRIPTION 

In a college-like setting, UBMS projects offer academic enrichment in math and science to 
improve student achievement in those subjects and to expose students to math and science 
careers. In this section, we describe the following features of UBMS projects in more detail: the 
setting in which these projects provide services; the goals, academic orientation, academic 
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offerings (summer and academic year) and instructional approaches of these projects; and the 
intensity and quantity of services the UBMS projects provide. 



1. Setting 

As described earlier, UBMS projects are typically hosted by two- and four-year 
postsecondary institutions (see Section A.l). Most UBMS projects are hosted by four-year 
colleges and universities, and most of these institutions have dormitories to house their students. 
These dormitories are often available in the summers to house participants of summer programs 
hosted by these institutions. 

UBMS projects typically exposed participants to a college setting during the summer 
program by housing them in the college dormitories. Virtually all the UBMS projects we 
surveyed (100 percent in 1994, 97 percent in 1998) offered a residential component to their 
summer programs compared with 87 percent of regular Upward Bound programs in 1992 
(Moore 1997). At almost all UBMS projects, students lived in the dormitories for the entire 
summer program, which lasted about six weeks on average. Only four percent of UBMS projects 
in 1994 had a residential component shorter than the summer program, but by 1998, the rate had 
increased to 1 1 percent. Therefore, for six weeks, most participants lived on campus like many 
college students do during the academic year. 



2. Goals of the Program 

As mentioned in Chapter I, the general objective of the UBMS program is to prepare 
participating students for postsecondary programs leading to careers in math and science. Seven 
out of 10 UBMS projects operating in 1994 rated “academic performance in math and science” 
as their foremost or second most important goal (see Exhibit II.7). The focus on academic 
improvement was similar to the focus of regular Upward Bound projects operating at about the 
same time. Eighty-seven percent of regular Upward Bound projects rated “academic 
improvement” as their foremost or second most important goal in 1993. If the regular Upward 
Bound grantee survey had also listed “academic improvement in math and science,” it is possible 
that some respondents would have cited that as one of their top two goals: Easciano and Jacobson 
(1997) characterized 37 percent of regular Upward Bound projects as having a strong emphasis 
on math and science. 

However, two goals that regular Upward Bound projects considered moderately important 
were not considered important by UBMS projects. Eirst, only 13 percent of UBMS projects 
reported that one of their top two goals was fostering students’ personal skills (e.g., goal 
orientation, ability to adapt to new settings), compared with 31 percent of regular Upward Bound 
programs. Second, none of the UBMS projects cited improving students’ access to financial aid 
as one of their top two goals, compared with 35 percent of regular programs. 
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3. Academic Orientation 



UBMS projects try to provide academic enrichment beyond what students are exposed to in 
school (see Exhibit II.8). Very few projects emphasized remedial instruction in 1994. By 1998, 
the relative focus on enrichment was even greater, with 83 percent of UBMS projects citing 
enrichment as a major emphasis and only 1 percent citing remediation. Although about 1 in 4 
regular Upward Bound programs emphasized providing remedial instruction, fewer than 1 in 10 
UBMS projects reported doing the same. 

The focus of UBMS on academic enrichment over remediation is consistent with the types 
of students served by the program. As described earlier, findings in Moore (1997) indicate that, 
on average, UBMS participants probably had less need for remedial support than regular Upward 
Bound participants. 



EXHIBIT II.7 

MOST IMPORTANT GOALS OF UPWARD BOUND MATH-SCIENCE PROJECTS, SUMMER 1994 



Academic improvement 
in math and science 

Exposure to careers in 
math and science 

Perseverance in math 
and science 

General academic 
improvement 



Exposure to college 

0% 20% 40% 60% 80% 100% 

Percent of UBMS Projects 

■ Most important □ Second most important 
Source: 1998 survey of Upward Bound Math-Science projects. 
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EXHIBIT II.8 



ACADEMIC ORIENTATION OE UPWARD BOUND MATH-SCIENCE PROJECTS 



Major Emphasis 


Upward Bound Math- 
Science, 

Summer 1994 (%) 


Regular Upward Bound, 
Summer 1992 (%)^ 


Support — instruction that parallels what students are 
taught in regular school courses 


33 


55 


Remediation — instruction that concentrates on 
fundamental concepts and skills that were taught in 
earlier grades 


6 


23 


Enrichment — instruction in concepts and material 
beyond what students are exposed to in regular school 
classes 


73 


69 



Source: 1998 survey of Upward Bound Math-Science projects; Nancy J. Easciano and Jon E. Jacobson, “Grantee 
Survey Report.” In A 1990s View of Upward Bound: Programs Offered, Students Served and Operational Issues, by 
U.S. Department of Education, Office of the Under Secretary (Washington, D.C.: U.S. Department of Education, 
May 1997). 

Note: Percentages do not total 100 because grantees were allowed to rate more than one approach as a major 
emphasis. 

t Excludes summer bridge programs for Upward Bound participants who have just graduated from high school. 



4. Summer Offerings 

In accordance with program guidelines, UBMS projects offered instruction in a diverse array 
of academic subject areas (see Exhibit II.9). Seventy-five percent or more of these projects in 
1994 offered instruction in the following subjects: writing and composition, algebra II, geometry, 
pre-calculus, computer applications and software use, physics, biology and chemistry. The 
average total number of offerings in 1994 was about 14, with a range of 2 to 22. The average 
number of offerings in math and science was about 7, with a range of 1 to 1 1. Thus, on average, 
math and science courses accounted for roughly half of UBMS projects’ total offerings. 

UBMS projects clearly differed from regular Upward Bound projects in their relative 
emphasis on certain subjects. First, as expected, they concentrated their offerings more on math 
and science. Although UBMS projects were no more likely than regular Upward Bound projects 
to offer certain math or science courses (for example, algebra II, geometry pre-calculus, calculus, 
biology and chemistry), UBMS projects were much less likely to offer instruction in areas 
outside of math and science, including social science or history courses or electives or 
nonacademic courses. Second, consistent with their greater emphasis on enrichment than on 
remediation, UBMS projects were less likely than regular Upward Bound programs to offer low- 
end courses such as reading comprehension and vocabulary, pre-algebra, and earth science (see 
Exhibit II.9). 
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To help prepare students for a postsecondary education and postcollegiate careers in math or 
science, UBMS projects also offered a range of support services and activities (see Exhibit 
11.10). Among the most common activities were field trips (e.g., to math or science facilities) and 
assistance with college and financial aid applications. The average number of these 
noninstructional offerings in 1994 was about 10, with a range of 3 to 15. 

UBMS projects were substantially less likely than regular Upward Bound projects to offer 
services focused on preparing for college. Many regular Upward Bound projects would have 
provided these services during the academic year: both regular Upward Bound and UBMS 
programs focus on academics during the summer. Because many UBMS participants participated 
in other precollege programs during the academic year, UBMS project staff members could have 
reasonably expected that those other programs were assisting students in preparing for college. 
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EXHIBIT II.9 



INSTRUCTION OFFERED BY UPWARD BOUND MATH-SCIENCE PROJECTS, BY SUBJECT AREA 





Upward Bound Math-Science 
Summer 1994 


Regular Upward Bound 
1992“ 


English/Language Arts 


Writing/Composition 


93% 


100% 


Literature 


60 


83 


Reading Comprehension and Vocabulary 


65 


98 


English as a Second Language 


13 


11 


Foreign Language 


54 


35 


Other 


9 


13 


Mathematics 


Pre-Algebra 


36 


82 


Algebra I 


69 


96 


Algebra II 


81 


95 


Geometry 


80 


95 


Pre-Calculus 


80 


80 


Calculus 


52 


58 


Statistics’’ 


17 


C 


Trigonometry’’ 


7 


C 


Other 


9 


24 


Computers 


Programming 


43 


47 


Applications/Software Use 


85 


79 


InternetAVeb Page Design’’ 


7 


C 


Other 


7 


6 


Science 


Physics 


76 


63 


Biology 


87 


89 


Chemistry 


81 


81 


Earth Science 


48 


66 


Other 


15 


19 


Social Science/History 


History 


11 


47 


Geography 


9 


24 


Sociology 


4 


17 


Psychology 


8 


15 


Government/Civics 


9 


40 


Other 


8 


13 


Electives/Nonacademic Courses 


Performing Arts 


31 


53 


Art 


26 


53 


Journalism 


28 


52 


Speech/Public Speaking 


48 


59 


Physical Eitness 


56 


69 


Other 


6 


26 



Sources: 1998 survey of Upward Bound Math-Science projects; Nancy J. Fasciano and Jon E. Jacobson, “Grantee Survey 
Report.” In A 1990s View of Upward Bound: Programs Offered, Students Served and Operational Issues, by U.S. 
Department of Education, Office of the Under Secretary (Washington, D.C.: U.S. Department of Education, May 1997), 39. 



Note: UBMS projects offer instruction in many areas besides math and science, either to meet regulatory requirements or 
simply to ensure that their program will interest and benefit students in many ways. 

1992 nonbridge summer programs or 1992-93 academic year. 

’’Neither survey listed statistics, trigonometry or InternetAVeb page design, but enough project directors 
specified them under “other courses” that we present data on these courses separately. 



19 




EXHIBIT II. 10 



NONINSTRUCTIONAL SERVICES OEEERED BY UPWARD BOUND MATH-SCIENCE PROJECTS 





Upward Bound Math- 
Science 
Summer 1994 


Regular Upward Bound 
1992" 


College Preparation/Skills 


Campus visits 


74% 


98% 


Adjusting to college living 


98 


92 


ACT/SAT preparation 


65 


97 


PSAT/PLAN or PACT preparation 


24 


73 


Help with financial aid or scholarships'’ 


80 


100 


Assistance with college applications 


78 


99 


Assistance with financial aid applications 


72 


100 


Career/Employment Assistance 


Site visits to employers” 


65 


59 


On-campus (employers or career representatives) 


63 


78 


Project-Related Work Experience 




49<i 


Job Training Partnership Act job 


0 


d 


Work- study job 


4 


d 


Math or science internships 


24 


d 


Job through other partnerships 


7 


d 


Field Trips to 


Academic science or math facilities 


98 


e 


Nonacademic science or math facilities 


94 


e 


Sites to conduct math- or science-related field 


85 


e 


work 


Other 


93 


e 



Sources: 1998 survey of UBMS projects; Nancy J. Easciano and Jon E. Jacobson, “Grantee Survey Report.” In 
A 1990s View of Upward Bound: Programs Offered, Students Served and Operational Issues, by U.S. 
Department of Education, Office of the Under Secretary (Washington, D.C.: U.S. Department of Education, 
May 1997), 54. 

“ 1992 nonbridge summer programs or 1992-93 academic year. 

’’ In the regular Upward Bound grantee survey, this item was phrased, “Identify sources of financial aid.” 

In the regular Upward Bound grantee survey, this item was phrased, “Site visit to employers or job 
shadowing.” 

^Although the regular Upward Bound grantee survey had three separate items about Job Training Partnership 
Act positions, work-study arrangements, and other partnerships, the results were reported only in the aggregate, 
and it did not ask about math or science internships (Easciano and Jacobson 1997, 54). 

The regular Upward Bound grantee survey asked about field trips of varying lengths, not the destinations. 



5. Academic- Year Offerings 

Although UBMS projects also provided services to students during the academic year in 
the mid-1990s, these services were minimal compared with UBMS summer services and 
were typically far less numerous and less intense than academic year services provided in 
regular Upward Bound. During the 1994-95 academic year, about one-third of UBMS 
projects provided tutoring or study sessions, and just more than half provided assistance with 
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college applications (see Exhibit II. 1 1). The average UBMS project provided about three 
types of these services during 1994-95. 



EXHIBIT II. 1 1 

ACADEMIC YEAR SERVICES OEEERED BY UPWARD BOUND MATH-SCIENCE PROJECTS, 1994-95 

College application help 
before 12th grade 

Workshops 

Individualized projects 

Tutoring or study 
sessions 

Employment or 
internship assistance 

Other 

0% 20% 40% 60% 80% 100% 

Percent of UBMS Projects 

Source; 1998 survey of Upward Bound Math-Science projects. 




By 1998-99, the percentage of UBMS projects regularly providing these services had increased 
substantially. For example, the percentage providing tutoring or study sessions rose from 81 
percent to 93 percent, and the percentage providing assistance with college applications rose 
from 52 percent to 78 percent. This increase probably reflects the establishment of more locally 
oriented UBMS projects. Not surprisingly, geographic proximity to their participating students 
influenced whether UBMS projects provided certain services during the academic year. UBMS 
projects were substantially more likely to provide tutoring and workshops during the academic 
year if a relatively large percentage of their participants lived in the same city or town as the 
program host. For example, more than three-quarters of UBMS projects with a relatively large 
percentage of participants from the local area (above the median) offered tutoring during the 
academic year versus less than half of UBMS projects with a relatively small percentage (below 
the median). 
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6. Instructional Approaches 

In 1994, the most common instructional approach taken by UBMS projects was the 
providing of instruction through courses in separate subjects. Four out of five UBMS projects 
offered courses in separate subjects (see Exhibit 11.12, Panel A). In three out of four UBMS 
projects, the primary method of instruction was either the provision of these courses (37 percent, 
see Exhibit 11.12, Panel B) or the combination of these courses with interdisciplinary courses 
(also 37 percent, see Exhibit 11.12, Panel B). 
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EXHIBIT II. 1 2 



INSTRUCTIONAL METHODS USED BY UPWARD BOUND MATH-SCIENCE PROJECTS, 1994 



A. Use of Different Instructional Methods 



Separate subjects 



Interdisciplinary courses 



Large project 



Other 



0% 20% 40% 60% 80% 100% 

Percent of DBMS Projects 



B. Primary Method Used 



Separate subjects 
and 

interdisciplinary 
courses 
37% 



Other 
6 % 

courses 

9% 

Source; 1998 survey of Upward Bound Math-Science projects. 

Note: Percentages may not total 100 percent due to rounding. 
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However, UBMS projects frequently used other instructional methods. The majority of 
projects (63 percent) offered interdisciplinary courses, and in a large minority (35 percent), at 
least some students worked on a large project or experiment that spanned multiple academic 
subject areas (see Exhibit 11.12, Panel A). 

UBMS projects vary considerably in how they sort students into classes or groups. In the 
summer of 1994, about half of the projects placed their students in instructional groups based on 
proficiency level (37 percent) or grade level (16 percent). About one-fourth placed students with 
diverse proficiency levels in the same group to facilitate learning (presumably the learning of 
less proficient students), and the remaining projects grouped students by their interests or in 
some other way. 

UBMS participants do not spend most of their time in traditional lecture-style classes. At the 
average project during the summer of 1994, only one-fourth of the time was spent in lecture-style 
classes such as those offered in most schools. The remaining time was spent in small-group, 
teacher- led instruction (32 percent), laboratories (29 percent), computer-based instruction (12 
percent), and other settings (4 percent). 



7. Intensity and Quantity of Services 

The services that UBMS projects offer and the length of their summer residential summer 
programs suggest that these projects offer intensive programs that provide students with a “large 
dose” of services, at least for one summer. Furthermore, larger doses of effective services may 
yield larger impacts than smaller doses, as we found for regular Upward Bound (Myers et al. 
2004). Summary measures of program intensity presented in this section indicate that UBMS 
projects offer an intensive program that might be expected to improve the math and science 
preparation of program participants. 

UBMS is a resource-intensive program. Program grants to UBMS projects provided an 
average of approximately $4,800 per student in FY 2004 (see Exhibit 11.13). This level of 
funding is comparable with funding for regular Upward Bound — approximately $4,500 per 
student — and much more expensive than other precollege programs. UBMS funding supports an 
extensive package of instruction and services, as described earlier in the chapter. 

Participants devote a substantial amount of time to the program, and most of this time is 
spent on academics. At the average UBMS project in the summer of 1994, students spent about 
29 hours per week receiving instruction and almost 1 1 hours per week on tutoring and 
homework. Thus, participating in UBMS over the summer is somewhat like having a full-time 
job requiring 40 hours per week. Because the vast majority of UBMS summer programs last six 
weeks, participants at the average project spend 240 total hours on academics during the 
summer. Only five UBMS projects reported a program of a different length: three had a five- 
week session, one had a seven-week session, and one had an eight-week session. 

The amount of time devoted to core program activities varied across UBMS projects. For 
example, four projects reported that students spent more than 40 hours per week in instruction 
alone, while a handful of projects reported estimates of less than 10 hours per week. Estimates of 
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time spent on tutoring or homework ranged from 0 to 20 hours per week. And the estimates of 
average total time spent on academies during the summer varied considerably, from fewer than 
100 hours at some projects to more than 340 hours at some others. 

Most UBMS participants completed the summer program. At the average project in the 
summer of 1994, 94 percent of participants completed all the requirements of the program. A 
few projects reported completion rates of 50 percent or less, but this information does not 
necessarily indicate a high dropout rate. Students might have attended the summer program for 
its full length but be counted as failing to “complete all the requirements of the program” (the 
wording in the questionnaire) because they did not, for example, turn in all their assignments. 



EXHIBIT II. 1 3 

PER-CAPITA PENDING POR REGULAR UPWARD BOUND AND POR UPWARD BOUND MATH-SCIENCE 

(UBMS) PROJECTS, PY 2004 



Regular Upward 
Bound 



UBMS 




$0 



$2,000 $4,000 



$6,000 



Note; Regular Upward Bound cost per participant listed in January 2010 ($4,721) is slightly above that listed 

in PY2004 U.S. Department of Education program data Web pages presented in the figure above. 

Source: U.S. Department of Education program data Web pages 

http://www.ed.gov/ pro grams/trioupbound/ funding.html and 
http://www.ed.gov/programs/triomathsci/funding.html (accessed January 2010). 



Some students participate for multiple summers or receive services during the academic 
year. As indicated earlier, about 70 percent of UBMS projects operating in summer 1994 allowed 
students to return from the previous summer. Among those projects, an average of 35 percent of 
1994 participants had also participated during 1993. In summer 1998, even more UBMS projects 
(more than 90 percent) allowed students to return; moreover, at those projects an average of 52 
percent of students had also participated in summer 1997. Furthermore, some UBMS projects 
extended students’ exposure to the program beyond the summer by providing services during the 
academic year. 
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III. THE IMPACTS OF THE UPWARD BOUND MATH-SCIENCE PROGRAM 



UBMS was established to increase economically disadvantaged students’ achievement in 
high school math and science courses, to increase the likelihood that they would major in math 
and science in college, and ultimately to increase their representation in math and science 
careers. Until now, no rigorous studies have measured the extent to which the program achieves 
its goals. In this chapter, we assess the effects of UBMS on important outcomes for program 
participants seven to nine years after they were scheduled to graduate from high school. 

In the remainder of this section, we describe the study design, the data sources and 
outcomes, as well as the results of the impact analysis. 



A. STUDY DESIGN 

UBMS provides intensive academic enrichment in math and science. Like regular Upward 
Bound projects, most UBMS projects also offer some assistance in preparing for college, 
including assistance with college applications (see Chapter II for more details). The combination 
of intensive academic enrichment in math and science with college preparation assistance 
suggests that UBMS might have positive effects on the following outcomes for program 
participants: (a) performance in high school, especially in math and science courses; (b) 
postsecondary attendance, persistence and completion; and (c) the likelihood of completing a 
postsecondary degree in a math and science field. Therefore, we designed the analysis to answer 
the following three research questions: 



1. What are the effects of UBMS participation on student performance in high school 
overall and in math and science courses in particular?^ 

2. What are the effects of UBMS participation on college attendance, attendance at 
different types of colleges and universities, persistence, and college completion? 

3. What are the effects of UBMS participation on the likelihood of enrolling and 
completing a degree in math or science? 

4. How are these findings influenced by prior participation in regular Upward Bound? 



We have designed the analysis to measure the impacts of UBMS on two important 
subgroups: students who had previously participated in regular Upward Bound and students who 
had not participated. UBMS participants who previously participated in regular Upward Bound 



3. The interim report (Olsen et al. 2007) presented detailed findings on the effects of UBMS on high school 
outcomes. By the 1999-2000 data collection on which that report was based, nearly all sample members had 
progressed out of high school and into college. Therefore, this report focuses only on postsecondary outcomes. 
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may have received a large dose of precollege services and academic preparation before 
participating in UBMS. However, most of the other UBMS participants entered UBMS without 
having received such intensive services. On the one hand, it is reasonable to expect UBMS to 
have larger effects on the students who had not previously received intensive services: students 
who have already received them may have already received the boost they needed to succeed. On 
the other hand, UBMS participants may be better prepared to benefit from their participation if 
they have previously participated in intensive services. In our analysis, we compute the effects 
separately for students who had previously participated in regular Upward Bound and students 
who had not participated to assess whether the effects of UBMS depend on the amount of 
precollege services students have received to that point. 

Regular Upward Bound is just one of the other programs in which UBMS participants could 
have participated. Appendix A of Moore (1997) includes a list of 28 alternative math and science 
precollege programs. Annual per student costs were available for 17 of the 28 alternative math 
and science programs; of these 17, only two were more expensive per student-year than UBMS. 

It is important to clarify that we have not attempted to measure the effects of UBMS versus 
an absence of precollege services. The analysis was designed to measure the effects of 
participating in UBMS relative to what students would have participated in otherwise — which 
might include regular Upward Bound — not the effects of participating in UBMS relative to no 
program participation. Most UBMS participants in our sample did not participate in regular 
Upward Bound, and only a few other precollege programs are as intensive as Upward Bound. 
Therefore, most UBMS participants would have participated in less intensive precollege services 
if they had not participated in UBMS. 

In this section, we describe the design of the analysis used to measure the impacts of UBMS. 
The impact analysis is based on a matched comparison group that attempts to reduce two types 
of bias common to many nonexperimental studies: selection bias and bias attributable to different 
data collection protocols for the participant and comparison groups. The strength of the impact 
analysis rests on three features of its design: 



1. How we selected our initial samples, particularly the comparison sample 

2. How we collected “baseline” (preprogram) information on the two samples 

3. How we used that information to select a matched comparison group 

In the remainder of this section, we describe these three features of the study and our approach to 
estimating the effects of UBMS on student participants. 



1. Selecting the Samples 

For the impact analysis, we obtained our sample of program participants from the projects 
themselves. In 1998, we contacted the 62 Upward Bound MSCs that were operating at that time 
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and that had been operating between 1993 and 1995. Although we would have been interested in 
obtaining lists of students from projects that were no longer operating in 1998, we believed that 
it would be very difficult to obtain such lists. If MSCs that closed before 1998 operated less 
effective programs than those that remained open, then the results presented in this chapter may 
overstate the effectiveness of MSCs operating between 1993 and 1995. From the MSCs that we 
contacted, we requested lists of the students who had participated in their program in the summer 
of 1993, the summer of 1994 or the summer of 1995; we received participant lists from all but 
one of these MSCs. To reduce the costs of collecting the necessary data, we randomly selected 
one out of every four of the students from these lists for our analysis sample. 

A primary feature of any nonexperimental evaluation is the choice of a comparison group. 
Experiments yield the best comparison groups because differences in outcomes between treated 
and untreated cases cannot be attributed to selection bias. In the absence of an experiment, the 
strength of an evaluation depends on the comparability between the participant and comparison 
groups. 

The most convenient comparison group for the impact analysis is also a compelling one — 
and the one we used. For the comparison group, we selected students from the evaluation of 
regular Upward Bound who reported that they had not participated in an MSC."^ In doing, so, we 
selected a comparison group with three desirable attributes: 



1 . Like UBMS participants, comparison students applied to participate in regular 
Upward Bound. Therefore, both UBMS participants and comparison students 
revealed a high level of motivation to pursue precollege services through Upward 
Bound or UBMS. This attribute provides some protection against a common source 
of selection bias in nonexperimental studies — bias from comparing more motivated 
participants with less motivated nonparticipants. 

2. Like UBMS participants, comparison students met the federal eligibility 
requirements to participate in some type of Upward Bound program — either 
regular or math and science. The federal eligibility requirements are the same for 
regular Upward Bound and for UBMS. To be included in the sample for the 
evaluation of regular Upward Bound — the sample from which we selected our 
comparison group — a student must have applied to a regular Upward Bound project 
and been determined eligible to participate. Therefore, both UBMS participants and 
comparison students in our sample must have come from either “low-income” 



4. If UBMS projects admitted eligible applicants on a fairly random basis, rejected applicants would probably 
constitute the best comparison group. However, MSCs are not required to select randomly from eligible applicants, 
and the evidence suggests that they do not. Case studies of 14 MSCs in the mid-1990s suggest that many MSCs 
impose additional eligibility requirements beyond the federal requirements, including an interest in math and science 
(Moore 1997). Therefore, it is unlikely that rejected applicants would have the virtues of a randomly selected control 
group for the purposes of this evaluation. Furthermore, the difficulty in obtaining lists of program participants from 
MSCs several years after they participated in UBMS suggests that many MSCs would not have been able to provide 
information on rejected eligible applicants. 
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families (income below 150 percent of the poverty line) or potential “first- 
generation” families (neither parent had earned a bachelor’s degree). 

3. Like UBMS participants, many comparison students would have met project- 
specific eligibility requirements imposed by some MSCs. Moore (1997) indicates 
that MSCs often apply additional admissions criteria in selecting applicants — criteria 
that include a minimum GPA in math and science. In this chapter, we show that 
many comparison students met the same criteria: many were successfully “matched” 
to UBMS participants who took similar courses and earned similar grades in math 
and science in ninth grade. 



Although the regular Upward Bound sample is a useful comparison group for measuring the 
effects of UBMS, it is not a perfect one. Data from the 1990s suggest that MSCs typically had 
more stringent minimum GPA requirements than regular Upward Bound projects. In addition, 
MSC staff members — who typically had recent experience with regular Upward Bound 
participants — reported that UBMS participants tended to be “more serious about school” than 
regular Upward Bound participants (Moore 1997, 26). Therefore, UBMS participants might fare 
better than regular Upward Bound participants, even without any assistance from UBMS, and 
simple differences in mean outcomes between the UBMS participant and comparison groups 
may overstate the effects of UBMS due to selection bias. 

To reduce selection bias, we selected a matched comparison sample from the regular 
Upward Bound sample. More specifically, we matched each UBMS participant to one or more 
regular Upward Bound sample members with similar characteristics based on data collected from 
student surveys and transcripts. Very few students in the regular Upward Bound sample 
participated in UBMS, and those who reported participating in UBMS were excluded from the 
comparison group. See Section 3 in this chapter for more details. 



2. Collecting Baseline Data 

The baseline variables constructed for the impact analysis characterize members of the two 
samples early in high school before UBMS participants in our sample entered UBMS. These 
variables are critical because they allow us to account — through a combination of matching and 
regression adjustments — for many preexisting differences between the two groups that might 
otherwise bias our impact estimates. To collect the information necessary to create baseline 
variables, we conducted student surveys and collected high school transcripts (see Appendix A). 

An important strength of the study’s design is that the data collection strategy was similar 
for both samples. Heckman, Smith and Clements (1997) argue that different survey 
questionnaires for the participant and comparison samples can be an important source of bias in 
nonexperimental studies. In this evaluation, we developed the initial survey questionnaire for 
UBMS participants from the baseline survey questionnaire for the regular Upward Bound 
sample. Baseline characteristics were collected for comparison group members through the 
baseline survey for the evaluation of regular Upward Bound. Baseline information on many of 
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the same characteristics was collected for UBMS participants through a follow-up survey 
conducted in 1999. Therefore, the survey questions from which we constructed baseline 
variables were often identical and always similar for the two samples. 

Even though the treatment and comparison groups were enrolled in high school at the same 
period of time, we had to address two differences in the surveys in constructing baseline 
variables for the MSC impact analysis: (a) the two survey questionnaires were different, and 

(b) the surveys were conducted at different times — after high school for the participant group and 
early in high school for the comparison group. To address the differences in the survey 
questionnaires, we used data from survey questions that are either identical or almost identical. 

To address the difference in the timing of the surveys, we used data from survey questions only 
when the timing of the survey was unlikely to affect the answer to the question. Furthermore, the 
approach to collecting and coding high school transcripts was the same for the two samples. 
Therefore, it is unlikely that differences in the data between the two samples have biased the 
impact estimates presented in this chapter. 

The baseline variables for the impact analysis fall into the following three categories: 

(a) demographic and family characteristics, (b) participation in other precollege programs, and 

(c) ninth-grade academic achievement not only in math and sciences but also more generally (see 
Exhibit III.l). We believe that the measures of students’ ninth-grade academic achievement are 
critical to the strength of the study. Given the findings in earlier reports, it seems entirely 
possible that even among students with similar demographic and family characteristics, students 
who participated in UBMS might have a higher academic aptitude and interest in math and 
science than students who participated in regular Upward Bound. Therefore, we use information 
on ninth-grade courses taken and ninth-grade GPA — overall and specifically math and science — 
to control for differences between the two samples in early high school achievement. 
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EXHIBIT III. I 



BASELINE VARIABLES 


Category 


Variables 


Source 


Demographic and family 


Sex 

Race and ethnicity 

Census region 

Native English speaker 

Mother’s time in the United States 

Sibling participated in Upward Bound 

High school cohort 


Initial surveys 


Prior program participation 
Ninth-grade achievement 


Sample member participated in regular Upward Bound 

GPA in math and science 
GPA in other subjects 
Math course taken 
Science course taken 


Initial survey and project records 
High school transcripts 



Note: The initial surveys were conducted in 1999 for Upward Bound Math-Science (UBMS) participants and in 
1992-94 for comparison students in the regular Upward Bound sample. To identify UBMS participants who 
had previously participated in regular Upward Bound, we used responses to the 1999 initial survey; to identify 
comparison students who participated in regular Upward Bound, we used participation information provided 
by projects. We collected high school transcripts in 2000 and 2003 for UBMS participants and in 2000 and 
earlier years for comparison students. 



3. Selecting a Matched Comparison Sample for the Impact Analysis 

The data collected on UBMS participants and regular Upward Bound sample members are 
useful in identifying not only many similarities between the two groups but also some 
differences (see Exhibit III.2). In both groups, 37 percent of the students were African American 
and 1 1 percent had a sibling who had participated in Upward Bound. The two groups were also 
similar with respect to many other characteristics, including the percentage taking algebra in 
ninth grade. However, the two groups exhibit differences on several dimensions. For example, 
UBMS participants were more likely to be male and tended to have higher grades than regular 
Upward Bound sample members. 

To address possible selection bias, we selected a matched comparison group from the 
regular Upward Bound sample using propensity score matching methods. Many studies adjust 
for baseline differences of these types using standard covariance adjustments — that is, by 
controlling for these variables in a regression analysis. However, the differences between the 
UBMS participant sample and the regular Upward Bound sample are too large to expect 
covariance adjustment to be reliable. Regression adjustments are likely to be unreliable if the 
means of the propensity scores are more than half a standard deviation apart (Rubin 2002). For 
the participant sample, the mean and standard deviation of the propensity score are 0.58 and 
0.22, respectively; for the comparison sample, the mean and standard deviation of the propensity 
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score are 0.25 and 0.23, respectively. Therefore, the difference in mean propensity scores is more 
than one standard deviation, and regression adjustments alone are likely to be unreliable. 



The goal in matching was to select a matched comparison sample from the regular Upward 
Bound sample such that the distributions of the baseline variables for the UBMS participant 
sample and the matched comparison sample were similar. Matching was conducted separately 
for sample members who had previously participated in regular Upward Bound and for those 
who had not: 



• UBMS participants who had previously participated in regular Upward Bound were 
matched to members of the treatment group (those offered the opportunity to 
participate in Upward Bound) in the experimental evaluation of regular Upward 
Bound that was conducted as part of the National Evaluation of Upward Bound. 

• UBMS participants who had not previously participated in regular Upward Bound 
were matched to members of the control group (those not offered the opportunity to 
participate in Upward Bound) in the experimental evaluation of regular Upward 
Bound that was conducted as part of the National Evaluation of Upward Bound. 
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EXHIBIT III.2 



SUMMARY STATISTICS FROM THE BASELINE VARIABLES 



Characteristic (percentage unless otherwise 
noted) 


Upward Bound 
Math-Science 
Participants 


Regular Upward 
Bound Sample 


Matched Comparison 
Group 


Participated in Regular Upward Bound 


18 


55 *** 


18 


Female 


59 


'72 *** 


59 


Race and Ethnicity 


African American 


37 


37 


37 


White 


25 


24 *** 


30 


Hispanic 


18 


20 


16 


Other race 


20 


9 *** 


17 


Region 


Northeast 


11 


9 


12 


Midwest 


23 


J9 ** 


28 


South 


40 


45 ** 


35 


West 


25 


26 


24 


Entry to High School 


1991-92 


32 


25 *** 


28 


1992-93 


39 


49 *** 


37 


1993-94 


29 


26 


35 


Other Characteristics 


Native English speaker 


80 


g'7 *** 


86** 


Mother in United States most of her life 


79 


g'7 *** 


83* 


Siblings in Upward Bound 


11 


11 


12 


Ninth-Grade Math Course 


Lower than algebra 


16 


22 *** 


14 


Algebra 


55 


53 


59 


More than algebra 


29 


24 *** 


27 


Ninth-Grade Science Course 


Biology, chemistry or physics 


37 


26 *** 


37 


Ninth-Grade GPA (mean) 


Math and science 


2.69 


1.65 *** 


2.71 


Other subjects 


3.24 


2.64 *** 


3.25 



*/**/*** Significantly different from Upward Bound Math-Science participants at the 0.10/0.05/0.0 1 levels, 
respectively. 



Regular Upward Bound sample members that were matched to at least one of the 689 
UBMS participants were included in the matched comparison sample (988 of the 1,134 eligible). 
The propensity score matching and the impact analysis are restricted to sample members who 
entered high school between 1991 and 1993. Although some UBMS participant sample members 
entered high school before 1991 and after 1993, relatively few comparison sample members did 
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so. Furthermore, the high school cohort is related to the likelihood of participating in the 
program and to the outcomes of interest because earlier cohorts had more time to enter college 
and select a field of study by the time they were interviewed for the evaluation. Therefore, to 
protect the internal validity of the study, we focused the analysis on students who entered high 
school between 1991 and 1993. 

UBMS participants could be matched to more than one regular Upward Bound sample 
member, and regular Upward Bound sample members could be matched to more than one 
UBMS participant. To be matched, a pair of students must satisfy the following condition: the 
difference between matched students in the log odds of the propensity scores was less than 0.20 
times the standard deviation of the log odds. Using matching with replacement within a fixed 
radius, each UBMS participant had, on average, 23.1 comparison group members matched to it, 
and each comparison group member matched to 16.1 UBMS participants. The weights of each 
UBMS participant were distributed equally to all comparison group members to whom that 
participant was matched, and the weight for a comparison group member is equal to the sum of 
the portions of the weights that group member received from each participant to whom it was 
matched. 

Using matching procedures, we were able to select a matched comparison group that is 
highly similar to the sample of UBMS participants on many dimensions (see the last column of 
Exhibit III.2). Only two of the differences between the groups are statistically significant — the 
difference in the percentage of sample members who are native English speakers and the 
percentage whose mother has lived in the United States for all or almost all of her life. 
Eurthermore, given the number of baseline variables, two is a small number of significant 
differences: we would expect about two significant differences even if the differences between 
the two groups were purely random. Although there is no guarantee that matching removed all 
unmeasured differences between the two samples, matching removed differences on a broad 
range of baseline variables — differences that might otherwise bias the impact estimates. 



4. Estimating the Impacts of UBMS Participation 

To measure the effects of UBMS participation on participating students, we used a 
regression-based approach that allows us to (a) adjust for the small remaining differences 
between the UBMS participant sample and the matched comparison group and (b) increase the 
precision of our impact estimates. The regression models yield estimates of the effect of UBMS 
on students who participated in the program. We regressed each outcome on a set of control 
variables and an indicator of whether the student participated in UBMS. The control variables 
included the variables used in selecting the matched comparison group: prior participation in 
regular Upward Bound; siblings in Upward Bound; sex; race; ethnicity; mother’s native language 
and immigrant status; high school cohort; region of the country; and several variables describing 
academic achievement in ninth grade, including GPA (separately for math and science courses 
and for other courses), math course taken, and science course taken. Eor continuous variables 
such as number of college credits, we estimated linear regression models; for categorical 
outcomes such as whether the sample member pursued postsecondary studies in math or science, 
we estimated logistic regression models or “logit” models. In estimating standard errors, we 
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accounted for clustering by project and implemented the Taylor series linearization methods used 
by the SUDAAN statistical analysis software. 

We estimated the effects of UBMS for the entire sample. However, the effects of UBMS 
may depend on the amount of other precollege services received. Therefore, we present separate 
impact estimates for those who participated in regular Upward Bound and those who did not 
participate. 



B. DATA SOURCES AND OUTCOMES MEASURES 
1. Outcome Measures 

The outcomes for which impact estimates are presented in this report can be grouped into 
four areas: postsecondary enrollment, persistence, completion, and field of study. 

Postsecondary enrollment. We estimate the impacts of UBMS on enrollment at any type of 
postsecondary educational institution, along with the highest level of postsecondary institution 
attended as well as the selectivity of four-year colleges and universities attended. Highest level 
of enrollment was defined a.s four-year for sample members who attended a public or private, 
nonprofit, four-year college or university; two-year for sample members who attended a public 
or private, nonprofit, two-year college, but not a four-year college or university; and vocational 
for sample members who attended a for-profit institution, but no two- or four-year institution. 

Selectivity of four-year colleges and universities attended was measured by using school 
ratings from Barron’s Profiles of American Colleges (Barron’s Educational Series 2003). If a 
school was rated as “most competitive,” “highly competitive” or “very competitive,” we 
classified the school as more selective. If a school was rated as “competitive,” “less 
competitive,” “noncompetitive,” “special” or unrated, or was excluded from Barron ’s, we 
classified the school as less selective. According to the classification system, more selective 
colleges and universities generally accept less than 75 percent of applicants, and students at more 
selective institutions were generally in the top half of their high school class. Less selective 
postsecondary institutions generally admit more than 75 percent of their applicants. The values 
of the four-year college or university selectivity outcome variables are set to zero for sample 
members who did not attend a four-year college or university. 

Postsecondary persistence. To assess progress toward completing and attaining 
postsecondary credentials, we requested transcripts from the relevant institutions for the sample 
members who reported having attended them. Credits were tabulated by level of institution and 
field of study. 

Postsecondary completion. We estimate the impacts of UBMS on completion of any 
postsecondary credential and on the highest postsecondary credential (degree, certificate or 
license) earned. Highest credential was defined as a four-year degree for sample members who 
earned a bachelor’s degree or higher; a two-year degree for sample members who earned an 
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associate’s degree, but not a bachelor’s degree; and a certificate or license for sample members 
who earned a postsecondary certificate or license, but no higher degree. 

Field of study. To determine whether a sample member had pursued postsecondary studies 
in math and science, we asked sample members for their “most recent or intended field of study,” 
and we classified their responses according to the same classification system used in the National 
Science Foundation’s Scientists and Engineers Statistical Data System. Furthermore, for the 
analysis, we separated the social sciences from other math and science fields (which we refer to 
as “math or science”) beeause the objectives of the program are more closely tied to math and 
science than to social sciences. 

The fields classified as science and engineering were biological sciences, computer science, 
engineering, mathematics, physical sciences and technical fields. The fields classified as 
nonscience and engineering were agriculture, arts, business, education, clerical or legal 
assistance, communications, health-related fields, humanities, trade and industry, protective 
services, and consumer or personal services. A small number of fields reported by sample 
members could not be classified as either science and engineering or nonscience and 
engineering. (For more details about NSF’s classification of fields of study, see 
http://www.nsf.gov/sbe/srs/nsf99337/pdf/appa.pdf.) 

To measure these postsecondary outcomes, we use data from the fifth follow-up survey and 
from administrative records. We describe these different data sources below, along with their 
strengths and weaknesses in providing valid information for correctly measuring these outcomes 
of interest. 

2. Data Sources 

The analyses described in this report are based on information provided (a) by UBMS 
participants and comparison group members during the follow-up surveys and by the 
postsecondary institutions that they reported attending as well as (b) by two administrative data 
sources. 

Surveys and transcripts. UBMS sample members provided retrospective baseline 
information during the 1998-1999 survey, and similar information was obtained for comparison 
group members from their baseline questionnaire when they applied to Upward Bound. The 
estimates in this report rely substantially on data from the follow-up survey conducted in 2003- 
2004. This survey foeused on obtaining information from sample members about their 
postsecondary educational attainment. After the follow-up survey, we also collected transcripts 
from high schools and postsecondary educational institutions attended by sample members. 

Administrative data. We collected data from other sources that allow us to augment our 
survey and transcript data. The NSC collects enrollment and degree information from the 
majority of colleges and universities in the United States, enabling it to provide verification of 
these activities by institution and semester. The federal SFA records are based on the Free 
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Application for Federal Student Aid (FAFSA) filled out by most college aspirants, and include 
information on aid application and receipt of Pell funding. 



3. Construction of the Outcome Measures 

The data available from the follow-up surveys, the NSC and the SFA records were used to 
construct various outcome measures in three different ways: using only the fifth follow-up 
survey, using only administrative records, and blending data from the surveys and the 
administrative sources in different combinations. Because data from the NSC were available for 
a period of time after the fifth follow-up survey was completed, we constructed two versions of 
an outcome when data from the NSC records were used: one using all the information available 
from the NSC records (NSC Full) and the other using information available from the NSC by the 
end of calendar year 2004, when the fifth follow-up survey was complete (NSC Truncated). A 
more detailed discussion about the construction of various outcome measures using these 
different data sources is provided in Appendix B of Seftor, Mamun, and Schirm (2009). 

We used the different data sources because they have different relative strengths and 
weaknesses. In conducting the impact analysis for this report, our basic principle was to use the 
maximum amount of information that was available on the sample members. Although the 
follow-up surveys provided data on a broad range of outcomes, we faced the problem of not 
having data for survey nonrespondents, and the nonrespondents might be systematically different 
from respondents, potentially leading to nonresponse bias in our estimates. The NSC and the 
SFA data are two convenient resources to mitigate this problem because we could get 
information on both survey respondents and nonrespondents from these administrative records. 

However, these administrative sources have their own limitations. The NSC does not cover 
the entire universe of postsecondary schools and does not cover all member schools for the entire 
relevant time period. Nationally, current rates of coverage are 87 percent for students attending a 
two-year institution and 90 percent for students attending a four-year institution. The coverage 
rates were lower in earlier years (the NSC data go back to 1993-1994); in terms of total U.S. 
college enrollment, the NSC data rose from 57 percent in 1997 to 88 percent in 2002, with small 
increases in subsequent years. Thus, the NSC might be missing data for a sample member who 
attended and potentially completed his or her education at a postsecondary institution because 
the institution was not covered by the NSC during the relevant years. SFA records provide data 
on all sample members; however, they do not have information on postsecondary completion, 
and they provide information on enrollment for only some students (those who received a Pell 
grant). 



C. ANALYSIS AND FINDINGS 

As documented in detail in this section, UBMS affected the types of institutions participants 
attended, both in level and selectivity. UBMS increased enrollment at four-year and selective 
colleges, and some of this increase can be attributed to students who would otherwise have 
attended only two-year colleges. We found evidence that UBMS increased persistence and 
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completion rates, primarily through the eompletion of four-year degrees. In terms of field of 
study, we found that UBMS raised the likelihood of pursuing postsecondary studies and of 
completing a four-year degree in social science, but we found no detectable effeet that the 
program increased these outeomes in math and seienee fields. 



1. The Effect of UBMS on Postsecondary Attendance, Persistence and Completion 

Although UBMS focuses on preparing students to major in math and science and to 
complete a degree in a math and science field, a person must enroll in college before choosing a 
major and must complete college to earn a degree in a math and seienee field. Even if UBMS 
had little effeet on students’ ehoiee of major, UBMS might be a cost-effeetive strategy to 
increase college enrollment and completion for disadvantaged students. Therefore, we assessed 
whether UBMS promotes postsecondary attendance, persistence and completion before we 
examined its effeets on college major. 

Exhibit III. 3 presents the effect of the participation in UBMS on any postsecondary 
enrollment (four-year, two-year or other), using a variety of enrollment measures. We present 
findings on a range of measures, based on a number of data sources and assumptions; a detailed 
description of the measures can be found in Appendix A. Eaeh row in the exhibit presents the 
results from a separate analysis, using one version of the outcome measure; thus. Exhibit III.3 
presents 27 versions of the “any postsecondary enrollment” outcome. The “Data Source” column 
describes the sourees that were used in the construetion of the particular measure and, where 
relevant, the order of their use. The “Uneoded” column provides details on how we dealt with 
cases for which the data do not provide definitive evidence of enrollment. The remaining 
columns present the findings from the analysis, including the regression-adjusted UBMS 
partieipants mean, the eomparison group mean, the impaet of UBMS on the measure, an 
indieator for statistical significance, and the p-value underlying the signifieance indieator. 

As noted earlier, the evidence suggests that, on average, UBMS serves students who 
perform well in high sehool and attend college at higher rates than average low-income students. 
Estimates for the matched comparison group using the fifth follow-up survey suggest that 91 
percent of UBMS participants would have attended a postsecondary institution if they had not 
participated in UBMS, leaving little room for improvement because of UBMS partieipation. We 
find that the impaet of UBMS on overall postseeondary enrollment is unclear, with mixed results 
that are sensitive to the data source. More speeifically, we either could not compute an impact or 
found small negative effects when the data source included the NSC data because the 
postseeondary enrollment rate for the matehed eomparison group was 100 percent. 

However, underlying the cloudy picture for overall enrollment is a much clearer finding of 
significant effects of UBMS participation on the types of postseeondary institutions attended. 

The impact estimates were (a) all positive (ranging from 8.6 to 18.2, with an average of 12.0) 
and signifieant for attending a four-year institution; (b) all negative (ranging from -9.6 to -3.7, 
with an average of -6.06 and only one estimate not statistieally different from zero) for attending 
a two-year college, but not a four-year eollege or university; and (c) nearly all negative (ranging 
from -1.7 to 0.2, with an average of -1.0 and one estimate significant) for attending a school 
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other than a two- or four-year institution (Exhibits III.4 through III.6). In other words, it appears 
that some of the increase in four-year college enrollment can be attributed to students who would 
otherwise have attended two-year colleges. 

Our analysis by previous participation in the regular Upward Bound provides additional 
insight. Almost all impacts on attending a four-year institution are larger for UMBS participants 
without previous participation in regular Upward Bound (see Exhibit B.2 in Appendix B). 
Moreover, the overall negative effect on enrollment in a two-year college is driven almost 
exclusively by the impact for students who did not participate in Upward Bound (see Exhibit B.3 
in Appendix B). These findings suggest that UBMS increases four-year enrollment for both 
regular Upward Bound participants and nonparticipants, but leads to an additional shift from 
two-year to four-year enrollment for those who did not have prior Upward Bound participation. 

We also examined whether UBMS affected the level of selectivity of the four-year colleges 
and universities attended by participants. A substantial portion of UBMS participants who 
attended four-year institutions attended more- selective institutions, and the difference between 
treatment and comparison means is positive (ranging from 12.2 to 22.9, with an average of 18.6) 
and statistically significant for all of the alternative measures (Exhibit III.7). Our subgroup 
analysis by previous participation in the Regular Upward Bound confirms these findings (see 
Exhibit B.5 in Appendix B). 

Transcript information shows that the number of credits earned within seven to nine years 
after scheduled graduation for the comparison group was 83 and for the treatment group 95 
(Exhibit III. 8). This 12-point difference was not statistically significant in our analyses. 

Einally, data from survey responses was combined with information from the NSC to create 
measures of postsecondary completion at various types of schools. UBMS increased the 
likelihood of completing any postsecondary credential (ranging from 9.0 to 12.1, with an average 
of 11.0) (Exhibit III. 9). This increase in overall postsecondary completion is primarily 
attributable to an increase in the percentage of sample members whose highest credential was a 
bachelor’s degree (ranging from 11.8 to 16.8, with an average of 15.7, and statistically 
significant for all of the measures); the program decreases the likelihood of completing an 
associate’s degree (ranging from -5.0 to -2.7, with an average of -3.8, and most negative 
impacts significant). UBMS did not have a detectable effect on the likelihood of completing a 
certificate or license (ranging from -0.8 to 0.5, with an average of -0.1 and only one estimate 
significant). 



2. The Effect of UBMS on Postsecondary Field of Study 

A primary objective of UBMS is to prepare students for postsecondary studies in math and 
science. The evidence suggests that UBMS participation encourages students to pursue 
postsecondary studies in math, science or the social sciences. UBMS increased the likelihood of 
pursuing postsecondary studies in the social sciences from 8.2 percent to 12.8 percent overall and 
from 7.4 percent to 12.5 percent at four-year institutions, though neither is statistically 
significant. UBMS also increased the likelihood of majoring or intending to major in math or 
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science from 25.8 percent to 31.6 percent overall and from 20.2 percent to 24.7 percent at four- 
year colleges and universities, although neither of these impacts is statistically different from 
zero at conventional levels (see Exhibit III. 10). 

The study found that 8.2 percent of the comparison group majored or intended to major in 
social sciences, and 12.8 percent of the UBMS participants so reported. Among those at four- 
year institutions, 7.4 percent of the comparison group and 12.5 percent of the UBMS participants 
majored in the social sciences. Neither of these differences was statistically significant in our 
analyses. 

Considering math and sciences, the percentage majoring in or intending to major in these 
fields is 25.8 for the comparison group and 31.6 percent for the UBMS group overall. Again, 
neither of these differences was statistically significant. 
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EXHIBIT III.3 



IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON ANY POSTSECONDARY ENROLLMENT, 

PERCENTAGE OF STUDENTS 



Variables 


Data Source 


U ncoded 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig 


P-value 


1 


5th Follow-Up Survey (Survey) 


Set to Missing Value 


97.67 


91.25 


6.42 


*** 


0.00 


2 


NSC through 05-06 (NSCF) 


None 


99.76 


100.00 


-0.24 


*** 


0.00 


2T 


NSC through 03-04 (NSCT) 


None 


95.97 


95.91 


0.05 




0.97 


3 


Pell Receipt (SFA) 


None 


64.60 


67.98 


-3.38 




0.43 


4 


NSCF /SEA 


None 


99.88 


100.00 


-0.12 


*** 


0.00 


4T 


NSCT / SFA 


None 


97.76 


98.93 


-1.17 




0.31 


5A 


Survey /NSCF /SEA 


Set to 0 


99.88 


100.00 


-0.12 


*** 


0.00 


SAT 


Survey /NSCT / SFA 


Set to 0 


99.17 


99.68 


-0.51 




0.16 


5B 


Survey /NSCF /SEA 


Set to 0 if no aid app 


100.00 


100.00 


! 






5BT 


Survey /NSCT / SFA 


Set to 0 if no aid app 


99.59 


99.68 


-0.09 




0.80 


5C 


Survey /NSCF /SEA 


Set to Missing Value 


100.00 


100.00 


! 






5CT 


Survey /NSCT / SFA 


Set to Missing Value 


99.40 


99.78 


-0.38 




0.32 


6A 


Survey / SFA 


Set to 0 


90.73 


86.11 


4.62 




0.16 


6B 


Survey/ SFA 


Set to 0 if no aid app 


95.59 


89.06 


6.53 


** 


0.02 


6C 


Survey/ SFA 


Set to Missing Value 


97.89 


93.28 


4.61 


*** 


0.00 


7A 


Survey /NSCF 


Set to 0 


99.76 


100.00 


-0.24 


*** 


0.00 


7AT 


Survey /NSCT 


Set to 0 


98.91 


99.47 


-0.56 




0.20 


7C 


Survey /NSCF 


Set to Missing Value 


100.00 


100.00 


! 






7CT 


Survey /NSCT 


Set to Missing Value 


99.61 


99.73 


-0.12 




0.71 


8 


Survey then NSCF / SFA 


Set to 0 


97.95 


92.98 


4.97 


*** 


0.00 


8T 


Survey then NSCT / SFA 


Set to 0 


97.79 


92.88 


4.91 


*** 


0.00 


9A 


Mult Surveys (3rd- 5th) / NSCF / SFA 


Set to 0 


100.00 


100.00 


! 






9AT 


Mult Surveys (3rd- 5th) / NSCT / SFA 


Set to 0 


99.59 


99.68 


-0.09 




0.80 


9B 


Mult Surveys (3rd- 5th) / NSCF / SFA 


Set to 0 if no aid app 


100.00 


100.00 


! 






9BT 


Mult Surveys (3rd- 5th) / NSCT / SFA 


Set to 0 if no aid app 


99.59 


99.68 


-0.09 




0.80 


9C 


Mult Surveys (3rd- 5th) / NSCF / SFA 


Set to Missing Value 


100.00 


100.00 


! 






9CT 


Mult Surveys (3rd- 5th) / NSCT / SFA 


Set to Missing Value 


99.59 


99.68 


-0.09 




0.80 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = 
Student Financial Aid. Impact may not exactly equal the difference between Upward Bound Math-Science 
Participants and the Matched Comparison Group due to rounding. ! Impacts could not be assessed due to 
complete or semicomplete separation. All estimates are weighted to account for missing data. The comparison 
group estimates and impact estimates are regression adjusted (see Chapter III, Section A.4). See Exhibits B.l 
through B.8 in Appendix B for estimates by prior participation in Regular Upward Bound. 

**/*** Statistically significant at the 0.05/0.01 levels, respectively. 
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EXHIBIT III.4 



IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON HIGHEST LEVEL OF POSTSECONDARY 
ENROLLMENT: FOUR- YEAR INSTITUTION, PERCENTAGE OF STUDENTS 



Variables Data Source 


U ncoded 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig 


P-value 


1 


Sth FoUow-Up Survey ( Survey) 


Set to Missing Value 


88.65 


76.14 


12.51 


*** 


0.00 


2 


NSC through OS-06 (NSCF) 


None 


71.85 


53.83 


18.02 


*** 


0.00 


2T 


NSC through 03-04 (NSCT) 


None 


70.09 


52.59 


17.50 


*** 


0.00 


3 


Pell Receipt (SFA) 


None 


83.34 


73.01 


10.34 


*** 


0.00 


4 


NSCF /SEA 


None 


86.79 


74.32 


12.47 


*** 


0.00 


4T 


NSCT/SFA 


None 


85.99 


74.25 


11.74 


*** 


0.00 


5A 


Survey /NSCF /SEA 


Set to 0 


91.12 


78.57 


12.55 


*** 


0.00 


SAT 


Survey /NSCT /SEA 


Set to 0 


90.67 


78.49 


12.18 


*** 


0.00 


SB 


Survey /NSCF /SEA 


Set to 0 if no aid app 


91.11 


78.57 


12.54 


*** 


0.00 


SBT 


Survey /NSCT /SEA 


Set to 0 if no aid app 


90.76 


78.49 


12.26 


*** 


0.00 


SC 


Survey /NSCF /SEA 


Set to Missing Value 


91.1 1 


78.57 


12.54 


*** 


0.00 


SCT 


Survey /NSCT /SEA 


Set to Missing Value 


90.79 


78.57 


12.22 


*** 


0.00 


6A 


Survey / SFA 


Set to 0 


89.44 


78.31 


11.13 


*** 


0.00 


6B 


Survey / SFA 


Set to 0 if no aid app 


89.58 


78.65 


10.92 


*** 


0.00 


6C 


Survey / SFA 


Set to Missing Value 


91.70 


82.38 


9.32 


*** 


0.00 


7A 


Survey / NSCF 


Set to 0 


84.5 3 


69.72 


14.81 


*** 


0.00 


7AT 


Survey / NSCT 


Set to 0 


83.70 


69.02 


14.68 


*** 


0.00 


7C 


Survey / NSCF 


Set to Missing Value 


84.75 


69.72 


15.03 


*** 


0.00 


7CT 


Survey / NSCT 


Set to Missing Value 


84.3 3 


69.20 


15.13 


*** 


0.00 


8 


Survey then NSCF / SEA 


Set to 0 


91.12 


78.57 


12.55 


*** 


0.00 


8T 


Survey then NSCT / SEA 


Set to 0 


90.67 


78.49 


12.18 


*** 


0.00 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


89.84 


80.90 


8.94 


*** 


0.00 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


89.3 8 


80.83 


8.55 


*** 


0.00 


9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid app 


89.84 


80.90 


8.94 


*** 


0.00 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid app 


89.3 8 


80.83 


8.55 


*** 


0.00 


9C 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


89.84 


80.90 


8.94 


*** 


0.00 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


89.38 


80.83 


8.55 


*** 


0.00 



Source: Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = 
Student Financial Aid. Impact may not exactly equal the difference between Upward Bound Math-Science 
Participants and the Matched Comparison Group due to rounding. All estimates are weighted to account for 
missing data. The comparison group estimates and impact estimates are regression adjusted (see Chapter III, 
Section A.4). See Exhibits B.l through B.8 in Appendix B for estimates by prior participation in Regular 
Upward Bound. 

*** Statistically significant at the 0.01 level. 
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EXHIBIT III.5 



IMPACT OF UPWARD BOUND MATHSCIENCE (UBMS) ON HIGHEST LEVEL OF POSTSECONDARY 
ENROLLMENT: TWO-YEAR INSTITUTION, PERCENTAGE OF STUDENTS 



Variables 


Data Source 


U ncoded 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig 


P-value 


1 


5th Follow-Up Survey (Survey) 


Set to Misang V alue 


7.781 


11.564 


-3.783 




0.113 


2 


NSC through 05-06 (NSCF) 


None 


6.611 


15.928 


-9.316 


*** 


0.000 


2T 


NSC through 03-04 (NSCT) 


None 


6.309 


15.941 


-9.632 


*** 


0.000 


3 


Pell Receipt ( SFA) 


None 


3.197 


8.695 


-5.498 


*** 


0.001 


4 


NSCF /SEA 


None 


3.934 


10.326 


-6.392 


*** 


0.002 


4T 


NSCT / SFA 


None 


4.249 


10.289 


-6.040 


*** 


0.003 


5A 


Survey /NSCF / SFA 


Set to 0 


4.563 


10.257 


-5.695 


*** 


0.006 


SAT 


Survey /NSCT /SEA 


Set to 0 


4.877 


10.228 


-5.351 


*** 


0.009 


5B 


Survey /NSCF / SFA 


Set to 0 if no aid app 


4.569 


10.257 


-5.688 


*** 


0.006 


5BT 


Survey /NSCT /SEA 


Set to 0 if no aid app 


4.767 


10.228 


-5.461 


*** 


0.008 


5C 


Survey /NSCF / SFA 


Set to Missing V alue 


4.569 


10.257 


-5.688 


*** 


0.006 


5CT 


Survey /NSCT /SEA 


Set to Missing V alue 


4.768 


10.238 


-5.470 


*** 


0.008 


6A 


Survey / SFA 


Set to 0 


4.953 


9.292 


-4.339 


** 


0.019 


6B 


Survey / SFA 


Set to 0 if no aid app 


4.755 


9.020 


-4.265 


** 


0.021 


6C 


Survey / SFA 


Set to Missing V alue 


4.822 


9.447 


-4.625 


** 


0.015 


7A 


Survey /NSCF 


Set to 0 


6.364 


13.711 


-7.347 


*** 


0.003 


7AT 


Survey /NSCT 


Set to 0 


6.670 


14.048 


-7.378 


*** 


0.002 


7C 


Survey /NSCF 


Set to Missing V alue 


6.386 


13.711 


-7.326 


*** 


0.003 


7CT 


Survey /NSCT 


Set to Missing V alue 


6.711 


14.085 


-7.374 


*** 


0.002 


8 


Survey then NSCF / SFA 


Set to 0 


4.563 


10.257 


-5.695 


*** 


0.006 


8T 


Survey then NSCT / SFA 


Set to 0 


4.877 


10.228 


-5.351 


*** 


0.009 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


4.443 


10.602 


-6.159 


*** 


0.003 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


4.755 


10.573 


-5.818 


*** 


0.004 


9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid app 


4.443 


10.602 


-6.159 


*** 


0.003 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid app 


4.755 


10.573 


-5.818 


*** 


0.004 


9C 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing V alue 


4.443 


10.602 


-6.159 


*** 


0.003 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing V alue 


4.755 


10.573 


-5.818 


*** 


0.004 



Source: Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = 
Student Financial Aid. Impact may not exactly equal the difference between Upward Bound Math-Science 
Participants and the Matched Comparison Group due to rounding. All estimates are weighted to account for 
missing data. The comparison group estimates and impact estimates are regression adjusted (see Chapter III, 
Section A.4). See Exhibits B.l through B.8 in Appendix B for estimates by prior participation in Regular 
Upward Bound. 

**/*** Statistically significant at the 0.05/0.01 levels, respectively. 
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EXHIBIT III.6 



IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON HIGHEST LEVEL 
OF POSTSECONDARY ENROLLMENT: OTHER INSTITUTION, PERCENTAGE OF STUDENTS 



Variables 


Data Source 


Uncoded 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig P-value 


1 


Sth Follow-Up Survey (Survey) 


Set to Missing Value 


1.08 


2.78 


-1.70 


* 0.09 


2 


NSC through OS-06 (NSCF) 


None 


0.90 


0.86 


0.04 


0.95 


2T 


NSC through 03-04 (NSCT) 


None 


1.04 


0.84 


0.20 


0.79 


3 


Pell Receipt ( SFA) 


None 


1.62 


2.83 


-1.22 


0.21 


4 


NSCF /SEA 


None 


1.64 


2.65 


-1.01 


0.30 


4T 


NSCT / SFA 


None 


1.73 


2.62 


-0.90 


0.36 


5A 


Survey /NSCF / SFA 


Set to 0 


1.68 


2.60 


-0.92 


0.34 


SAT 


Survey /NSCT /SEA 


Set to 0 


1.80 


2.57 


-0.77 


0.42 


SB 


Survey /NSCF / SFA 


Set to 0 if no aid app 


1.68 


2.60 


-0.92 


0.34 


SBT 


Survey /NSCT /SEA 


Set to 0 if no aid app 


1.80 


2.57 


-0.77 


0.43 


SC 


Survey /NSCF / SFA 


Set to Missing Value 


1.68 


2.60 


-0.92 


0.34 


SCT 


Survey /NSCT /SEA 


Set to Missing Value 


1.80 


2.58 


-0.77 


0.42 


6A 


Survey / SFA 


Set to 0 


1.60 


2.94 


-1.35 


0.16 


6B 


Survey / SFA 


Set to 0 if no aid app 


1.54 


2.97 


-1.43 


0.15 


6C 


Survey / SFA 


Set to Missing Value 


1.61 


3.11 


-1.51 


0.15 


7A 


Survey/NSCF 


Set to 0 


1.29 


2.72 


-1.43 


0.11 


7AT 


Survey /NSCT 


Set to 0 


1.45 


2.71 


-1.25 


0.16 


7C 


Survey/NSCF 


Set to Missing Value 


1.29 


2.72 


-1.43 


0.11 


7CT 


Survey /NSCT 


Set to Missing Value 


1.46 


2.72 


-1.25 


0.16 


8 


Survey then N SCF / SFA 


Set to 0 


1.68 


2.60 


-0.92 


0.34 


8T 


Survey then NSCT / SFA 


Set to 0 


1.80 


2.57 


-0.77 


0.42 


9A 


Mult Surveys (3rd-5th) / NSCF / S 


Set to 0 


1.67 


2.67 


-1.00 


0.30 


9AT 


Mult Surveys (3rd-5th) / NSCT / 5 


Set to 0 


1.79 


2.64 


-0.85 


0.38 


9B 


Mult Surveys (3rd-5th) / NSCF / S 


Set to 0 if no aid app 


1.67 


2.67 


-1.00 


0.30 


9BT 


Mult Surveys (3rd-5th) / NSCT / 5 


Set to 0 if no aid app 


1.79 


2.64 


-0.85 


0.38 


9C 


Mult Surveys (3rd-5th) / NSCF / S 


Set to Missing Value 


1.67 


2.67 


-1.00 


0.30 


9CT 


Mult Surveys (3rd-5th) / NSCT / 5 


Set to Missing Value 


1.79 


2.64 


-0.85 


0.38 



Source: Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = 
Student Financial Aid. Impact may not exactly equal the difference between Upward Bound Math-Science 
Participants and the Matched Comparison Group due to rounding. All estimates are weighted to account for 
missing data. The comparison group estimates and impact estimates are regression adjusted (see Chapter III, 
Section A.4). See Exhibits B.l through B.8 in Appendix B for estimates by prior participation in Regular 
Upward Bound. 

* Statistically significant at the 0. 1 level. 
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EXHIBIT III.7 



IMPACT OF UPWARD BOUND MATH-SCIENCE ON ATTENDANCE AT A HIGHLY-SELECTIVE 
FOUR- YEAR POSTSECONDARY INSTITUTION, PERCENTAGE OF STUDENTS 



Variables 


Data Source 


Uncoded 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig 


P-value 


1 


5th Follow-Up Survey (Survey) 


Set to Missing Value 


41.19 


27.09 


14.10 


*** 


0.01 


2 


NSC through 05-06 (NSCF) 


None 


36.26 


16.27 


19.99 


*** 


0.00 


2T 


NSC through 03-04 (NSCT) 


None 


35.05 


15.90 


19.15 


*** 


0.00 


3 


Pell Receipt ( SFA) 


None 


54.82 


32.75 


22.06 


*** 


0.00 


4 


NSCF /SEA 


None 


56.50 


33.73 


22.77 


*** 


0.00 


4T 


NSCT / SFA 


None 


55.95 


33.50 


22.46 


*** 


0.00 


5A 


Survey /NSCF / SFA 


Set to 0 


57.23 


37.30 


19.93 


*** 


0.00 


SAT 


Survey /NSCT /SEA 


Set to 0 


56.69 


37.07 


19.61 


*** 


0.00 


5B 


Survey /NSCF / SFA 


Set to 0 if no aid app 


57.31 


37.30 


20.01 


*** 


0.00 


5BT 


Survey /NSCT /SEA 


Set to 0 if no aid app 


56.82 


37.07 


19.75 


*** 


0.00 


5C 


Survey /NSCF / SFA 


Set to Missing Value 


57.31 


37.30 


20.01 


*** 


0.00 


5CT 


Survey /NSCT /SFA 


Set to Missing Value 


56.84 


37.11 


19.73 


*** 


0.00 


6A 


Survey / SFA 


Set to 0 


56.27 


36.90 


19.37 


*** 


0.00 


6B 


Survey / SFA 


Set to 0 if no aid app 


56.47 


37.00 


19.48 


*** 


0.00 


6C 


Survey / SFA 


Set to Missing Value 


59.54 


36.61 


22.93 


*** 


0.00 


7A 


Survey /NSCF 


Set to 0 


39.45 


26.68 


12.77 


** 


0.02 


7AT 


Survey /NSCT 


Set to 0 


38.61 


26.38 


12.23 


** 


0.02 


7C 


Survey /NSCF 


Set to Missing Value 


39.56 


26.68 


12.88 


** 


0.02 


7CT 


Survey /NSCT 


Set to Missing V alue 


38.92 


26.45 


12.47 


** 


0.02 


8 


Survey then NSCF / SFA 


Set to 0 


57.23 


37.30 


19.93 


*** 


0.00 


8T 


Survey then NSCT / SFA 


Set to 0 


56.69 


37.07 


19.61 


*** 


0.00 



Source: Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = 
Student Financial Aid. Impact may not exactly equal the difference between Upward Bound Math-Science 
Participants and the Matched Comparison Group due to rounding. All estimates are weighted to account for 
missing data. The comparison group estimates and impact estimates are regression adjusted (see Chapter III, 
Section A.4). See Exhibits B.l through B.8 in Appendix B for estimates by prior participation in Regular 
Upward Bound. 

**/*** Statistically significant at the 0.05/0.01 levels, respectively. 
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EXHIBIT III.8 



IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON TOTAL 
POSTSECONDARY CREDITS EARNED, PERCENTAGE OF STUDENTS 



Postsecondary Credits Earned 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig 


P -value 


Two- and four-year college and university 


95.088 


83.822 


11.267 




0.180 


T wo- year colleges 


9.507 


11.633 


-2.126 




0.359 


Four-year colleges 


83.441 


68.023 


15.418 


** 


0.031 


Math and Science 


29.514 


21.622 


7.892 


** 


0.037 



Source; Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: Impact may not exactly equal the difference between Upward Bound Math-Science Participants and the 
Matched Comparison Group due to rounding. All estimates are weighted to account for missing data. The 
comparison group estimates and impact estimates are regression adjusted (see Chapter III, Section A.4). See 
Exhibits B.I through B.8 in Appendix B for estimates by prior participation in Regular Upward Bound. 

** Statistically significant at the 0.05 level. 
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EXHIBIT III.9 



IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON COMPLETION OF ANY CREDENTIAL 
AND HIGHEST CREDENTIAL COMPLETED (INTENT TO TREAT), PERCENTAGE OF STUDENTS 



Variables 


Outcome / Data Source 


Uncoded 


UBMS 

Participants 


Comparison 

Group 


Impact 


Sig 


P- value 


Any Postsecondary Degree 














1 


Survey 


Set to Missing Value 


70.62 


60.78 


9.85 


* 


0.06 


2 


NSCF 


None 


47.51 


37.50 


10.01 


** 


0.01 


2T 


NSCT 


None 


44.28 


35.30 


8.98 


** 


0.03 


7A 


Survey / N SCF 


Set to 0 


66.10 


54.13 


11.97 




0.00 


7AT 


Survey /NSCT 


Set to 0 


63.85 


52.81 


11.04 


*** 


0.01 


7B 


Survey /NSCF /SFA 


Set to 0 if no aid app 


66.25 


54.13 


12.12 


*** 


0.00 


7BT 


Survey /NSCT /SFA 


Set to 0 if no aid app 


64.22 


52.90 


11.32 




0.01 


7C 


Survey / N SCF 


Set to Missing Value 


66.25 


54.13 


12.12 


*** 


0.00 


7CT 


Survey /NSCT 


Set to Missing Value 


64.24 


52.95 


11.29 


*** 


0.01 


Highest Degree Completed: Four-year Degree 














1 


Survey 


Set to Missing Value 


61.19 


45.00 


16.19 


** 


0.01 


2 


NSCF 


None 


45.87 


31.61 


14.25 


*** 


0.00 


2T 


NSCT 


None 


42.08 


30.27 


11.81 


** 


0.01 


7A 


Survey / N SCF 


Set to 0 


58.37 


41.76 


16.62 




0.00 


7AT 


Survey /NSCT 


Set to 0 


55.81 


40.65 


15.16 


*** 


0.00 


7B 


Survey /NSCF /SFA 


Set to 0 if no aid app 


58.50 


41.76 


16.75 


*** 


0.00 


7BT 


Survey /NSCT /SFA 


Set to 0 if no aid app 


56.13 


40.72 


15.41 




0.00 


1C 


Survey / N SCF 


Set to Missing Value 


58.50 


41.76 


16.75 




0.00 


7CT 


Survey /NSCT 


Set to Missing Value 


56.15 


40.76 


15.39 


*** 


0.00 


Highest Degree Completed: Two-year Degree 














1 


Survey 


Set to Missing Value 


6.27 


11.25 


-4.98 




0.03 


2 


NSCF 


None 


1.84 


5.63 


-3.79 


*** 


0.00 


2T 


NSCT 


None 


2.22 


4.87 


-2.66 


** 


0.03 


7A 


Survey / N SCF 


Set to 0 


5.17 


9.22 


-4.05 


** 


0.02 


7AT 


Survey /NSCT 


Set to 0 


5.46 


8.91 


-3.45 


* 


0.06 


7B 


Survey /NSCF /SFA 


Set to 0 if no aid app 


5.19 


9.22 


-4.03 


** 


0.02 


7BT 


Survey /NSCT /SFA 


Set to 0 if no aid app 


5.50 


8.92 


-3.42 


* 


0.06 


1C 


Survey / N SCF 


Set to Missing Value 


5.19 


9.22 


-4.03 


** 


0.02 


7CT 


Survey /NSCT 


Set to Missing Value 


5.50 


8.93 


-3.43 


* 


0.06 


Highest Degree Completed: Other Degree 














1 


Survey 


Set to Missing Value 


3.77 


4.53 


-0.76 




0.59 


2 


NSCF 


None 


0.52 


0.25 


0.26 




0.26 


2T 


NSCT 


None 


0.64 


0.15 


0.49 


* 


0.05 


7A 


Survey / N SCF 


Set to 0 


3.02 


3.15 


-0.13 




0.89 


7AT 


Survey /NSCT 


Set to 0 


3.08 


3.26 


-0.18 




0.86 


7B 


Survey /NSCF /SFA 


Set to 0 if no aid app 


3.03 


3.15 


-0.13 




0.90 


7BT 


Survey /NSCT /SFA 


Set to 0 if no aid app 


3.09 


3.26 


-0.18 




0.86 


1C 


Survey / N SCF 


Set to Missing Value 


3.03 


3.15 


-0.13 




0.90 


7CT 


Survey /NSCT 


Set to Missing Value 


3.09 


3.27 


-0.18 




0.86 



Source: Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = Student 
Financial Aid. Impact may not exactly equal the difference between Upward Bound Math-Science Participants and 
the Matched Comparison Group due to rounding. All estimates are weighted to account for missing data. The 
comparison group estimates and impact estimates are regression adjusted (see Chapter III, Section A.4). See Exhibits 
B.l through B.8 in Appendix B for estimates by prior participation in Regular Upward Bound. 
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*/**/*** Statistically significant at the 0.1/0.05/0.01 levels, respectively. 



48 




EXHIBIT III. 10 



IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON POSTSECONDARY FIELD OF STUDY 
(INTENT TO TREAT), PERCENTAGE OF STUDENTS 



Field of Study 


UBMS 

Participants 


Comparison 

Group 


Impact 


Field of Study at Most Recent PS Institution 

All PS institutions 


Math and science fields, excluding social sciences 


31.560 


25.766 


5.793 


Social science fields 


12.762 


8.238 


4.524 


Other fields 


46.800 


57.258 


-10.459 ** 


Four- year coUe^s and universities 


Math and science fields, excluding social sciences 


24.738 


20.248 


4.490 


Social science fields 


12.509 


7.359 


5.150 


Other fields 


42.718 


43.247 


-0.529 


Earned Degree or Certificate in Field 

All PS institutions 


Math and science fields, excluding social sciences 


15.310 


12.556 


2.753 


Social science fields 


8.570 


4.415 


4.155 


Other fields 


28.910 


31.189 


-2.279 


Four- year coUe^s and universities 


Math and science fields, excluding social sciences 


10.189 


6.372 


3.817 


Social science fields 


8.266 


3.975 


4.291 


Other fields 


20.931 


21.996 


-1.065 



Source: Mathematica Policy Research, Inc. analysis file “ITT_UBMS.sas.” 

Notes: PS indicates postsecondary. Impact may not exactly equal the difference between Upward Bound Math- 
Science Participants and the Matched Comparison Group due to rounding. All estimates are weighted to 
account for missing data. The comparison group estimates and impact estimates are regression adjusted (see 
Chapter III, Section A.4). See Exhibits B.I through B.8 in Appendix B for estimates by prior participation in 
Regular Upward Bound. 

*/** Statistically significant at the 0.1/0.05 levels, respectively. 
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IV. INTERPRETATION OF FINDINGS 



The estimated effects of UBMS paint a fairly consistent picture of UBMS improving 
participants’ postsecondary educational outcomes. Specifically, it increases enrollment at four- 
year and more selective institutions, increases math and science course taking, and increases 
postsecondary degree completion overall and at four-year institutions. 

Although we took several steps to reduce selection bias, it is certainly possible that the true 
effects of participating in UBMS are smaller than our estimates suggest. As reported earlier in 
the chapter, UBMS participants may be more serious about school than regular Upward Bound 
participants on average. This difference is reflected in our data: UBMS participants in our 
sample had higher GPAs and took more advanced math and science courses in ninth grade than 
members of the regular Upward Bound sample. We accounted for these differences by selecting 
a matched comparison group that resembled the UBMS participant sample in grades earned and 
in course taking in ninth grade. However, it is possible that despite earning similar grades and 
taking similar courses early in high school, the UBMS participant sample is somewhat more 
serious about school, more serious about math and science, or is different from the matched 
comparison group in some other way that would lead our analysis to overstate the effects of 
UBMS. 

Although we cannot measure the extent of selection bias, some informed speculation is 
helpful in interpreting the impact estimates. As we indicated earlier, students in both groups — the 
UBMS participant sample and the matched comparison group — exhibited some motivation to 
improve academically by applying to participate in Upward Bound. Therefore, motivational 
differences between the two groups are likely to be small and unlikely to bias the impact 
estimates. However, the impact estimates would overstate the true impacts if any of the 
following were true: 



• Members of the UBMS participant sample were higher achievers than members of 
matched comparison group. By the end of high school and after students in the 
UBMS sample had participated in UBMS, the average GPA was slightly but 
significantly higher for UBMS participants (3.14) than for matched comparison 
students (3.06). Even if the entire difference were attributable to selection bias 
instead of the effects of the program, the average student in each group was a B 
student. Therefore, if the UBMS participant sample contains higher achieving 
students than the matched comparison group, the difference seems to be small. 

• Members of the UBMS participant sample were better at math and science than 
members of the matched comparison group. If students in the UBMS participant 
sample were better at math and science than students in the matched comparison 
group, it is not reflected in their math and science course taking or in their grades 
during ninth grade: the two groups had similar grades in math and science, were 
equally likely to take algebra or geometry and were equally likely to take biology, 
chemistry or physics. Although we did not explicitly examine whether UBMS 
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participants were more likely to enroll in the advanced sections of courses before 
entering UBMS, other findings suggest that UBMS participants were no more likely 
to have taken Advanced Placement classes in math or science by the time they 
finished high school. Finally, focus groups conducted in 1996 suggest that UBMS 
participants do not view themselves as strong in math and science (Moore 1997, 28). 
Therefore, the information we have collected provides no reason to believe that when 
they entered UBMS, the participant sample was better at math and science than the 
matched comparison group. 

• Members of the UBMS participant group were more interested in math and 
science than members of the matched comparison group. None of the information 
that we extracted from student transcripts would suggest that the UBMS participant 
sample had greater interest in math and science. It is possible that UBMS 
participants had greater interest in careers in math and science that simply was not 
reflected in their high school course taking or in their grades early in high school. In 
1996 focus groups, many UBMS participants expressed interest in pursuing careers 
in scientific fields such as engineering, medicine, and nursing (Moore 1997, 28). 
However, many of these students indicated that their career interests had developed 
just that summer, and the expression of those interests could have been influenced by 
the fact that the focus groups were conducted on site at projects that emphasized 
math and science. 



We suspect that the impacts most vulnerable to selection bias are those that are most closely 
related to a person’s interest in pursuing careers in a math or science field. The ability to gauge 
students’ interest in math and science early in high school would probably require conducting 
assessments or survey interviews at that time. However, we first interviewed members of the 
UBMS participant sample after they had completed high school. The possibility that UBMS 
participants might have had greater interest in pursuing a career in science than matched 
comparison students raises the question of whether our estimates overstate the effects of UBMS 
on the outcomes that are most closely related to one’s career interests such as majoring in math 
or science in college. 

Although the findings in this report are promising, a note of caution is appropriate. We 
speculate that the selection bias is likely to be largest for outcome variables most closely tied to 
one’s interest in pursuing math and science careers, but it is not possible to measure the selection 
bias. Although we took several steps to reduce selection bias, the estimated effects of UBMS 
may overstate the true effects of the program. 

However, it is also worth noting that much of the impact of UBMS on postsecondary 
enrollment is through the type of institution attended, with a shift from two-year to four-year 
institutions. This finding was true for all UBMS participants, but was even stronger for those 
who had not participated in a regular Upward Bound program. The vast majority of UBMS 
projects are hosted by four-year institutions, and it is conceivable that part of the impact of 
UBMS is through exposure to that environment and its subsequent role in the enrollment 
decisions of participants. 
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APPENDIX A 

DATA COLLECTION AND OUTCOMES MEASURES 




This appendix describes and assesses the procedures for collecting the data that we used to 
measure the outcomes examined in this report. The data come from two types of sources: 



1 . Follow-up surveys of sample members 

2. Administrative records (National Student Clearinghouse and Student Financial Aid) 



This appendix focuses on procedures for obtaining completed interviews in the fifth follow- 
up survey, collecting administrative records, and constructing outcome measures from multiple 
sources. 



A. FIFTH FOLLOW-UP SURVEY OF STUDENTS 

The fifth follow-up survey was conducted between July 2003 and December 2004. It was 
designed to collect information on postsecondary outcomes. 



1. Data Collection Modes 

One week before we began interviewing, we sent a letter to all study participants. The letter 
stated that we would call them to complete an interview for an important study. In addition, the 
letter encouraged the individuals’ participation in the survey and noted that we would pay 
respondents $10 for completing the interview. Toward the end of the data collection, the U.S. 
Department of Education (ED) requested that the Office of Management and Budget approve an 
increase in the incentive to $25 in September 2004; the final 180 interviews were collected under 
the higher incentive scheme. 

Most interviews were administered using computer- assisted telephone interviewing (CATI). 
CATI interviews took about 30 minutes to complete. When a CATI interview was not possible, 
we attempted to obtain a completed questionnaire through the mail. Study participants also had 
the option of completing the survey on the Internet. Throughout the data collection process, we 
mailed questionnaires to study participants when respondents requested it or when we 
determined they could not be reached by telephone. We conducted three follow-up mailings after 
the first mailing, with the last set of questionnaires sent out in December 2004. 



2. Locating 

Throughout the data collection period, locating staff members used services such as 
LexisNexis and Internet databases to obtain updated addresses and telephone numbers for 
difficult-to-reach study participants. 
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3. Incentives 



To obtain a high response rate, we used financial incentives for survey completion. As 
noted, we offered study participants $10 or $25 for responding to the survey. We mailed 
incentive checks after the sample member completed the interview. 



4. Response Rates 

The eligible sample consisted of 687 UBMS participants (two were deceased) and 988 
sample members from the evaluation of regular Upward Bound. (See Chapter III for more details 
on the samples.) We obtained completed interviews for 534 UBMS participants and 804 regular 
Upward Bound sample members for response rates of 78 percent and 81 percent, respectively. 



B. TRANSCRIPT DATA COLLECTION 

For the fifth follow-up analysis of Upward Bound Math Science, we collected postsecondary 
transcripts between January 2004 and May 2006. We made transcript requests to institutions that 
were reported by sample members in either the fifth follow-up survey of sample members or 
earlier surveys. 



I. Preparation for Requesting Transcripts 

Information about students’ secondary and postsecondary enrollment was primarily obtained 
from follow-up interviews. Students reported the secondary and postsecondary institutions that 
they had attended. Secondary transcripts were requested only from UBMS sample members 
selected for the impact analysis; postsecondary transcripts were requested from all sample 
members — both UBMS and regular Upward Bound — who reported or confirmed having 
attended a particular postsecondary institution.^ 

To obtain mailing addresses for the schools that were attended by sample members, we 
matched schools that were reported by survey respondents to directories of secondary and 
postsecondary schools maintained by ED. Secondary schools were matched to the Common Core 
of Data; postsecondary schools were matched to the integrated Postsecondary Education Data 
System.^ 



1. We did not collect secondary transcripts for regular Upward Bound sample members because we had already 
collected these transcripts for a large percentage of the sample in previous waves of data collection. 

2. Students were asked to provide the name and state of each secondary and postsecondary school they attended, but 
sometimes misspellings or incomplete information resulted in some invalid requests for student transcripts because 
schools were matched with an incorrect address and transcripts were requested from the wrong school. When a 
school indicated that they could not fill a request because they had no record of the student whose transcript we 
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2. Procedures for Requesting Transcripts 



Each school was sent a transcript request packet that included the following: 



• A letter, printed on ED letterhead, which explained the purpose of the study and the 
reason we were requesting transcripts 

• A statement of Authorization and Confidentiality, which cited the Family 
Educational Rights and Privacy Act and included questions and answers regarding 
consent and confidentiality 

• A transcript checklist of all the materials that we requested from the school, 
including student transcripts, a course catalog, grade descriptions and a transcript 
reimbursement form, which would indicate the reimbursement that the school 
required for providing the requested transcripts 

• A postage-paid business reply envelope for sending the transcripts 

• A disclosure notice to be placed in each student’s file, indicating that a copy of his 
or her transcript was released to Mathematica Policy Research, Inc. (Mathematic a) 
as an agent to ED 



3. Follow-Up Procedures 

In instances when schools did not respond to our initial request for transcripts, we mailed 
another request for student transcripts. These mailings were done periodically as we tracked the 
schools that had not yet sent the requested transcripts and corrected requests that contained 
errors. 

As the targeted end date for collecting transcripts approached, interviewers started calling 
schools directly to inquire about the status of our requests. Many schools responded to these calls 
by faxing us the requested transcripts. When the school indicated that they could not provide one 
or more of the requested transcripts, the interviewer completed a problem sheet indicating the 
reason. The reason generally fell into one of the following categories: 

• The student was never enrolled at the school according to the school’s records. 

When this situation occurred, our first response was to call the school and provide 
more information on the student (e.g., provide or verify date of birth and dates of 
attendance) to see whether a transcript could be located with additional 
information. In many cases, the school was able to locate and provide transcripts 



(continued) 

requested, it was sometimes because of such mismatches. In these cases, we attempted to learn the correct name and 
address of the school where the student was enrolled and make a new transcript request. 
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once additional information was provided. In other cases, the school provided some 
information that helped us determine where we might obtain the needed 
transcripts. If the school had no record of the student having ever attended and we 
were unable to obtain additional information, then we marked the case as an invalid 
request. 

• Transcripts were held by the school district. Some schools held only the 
transcripts of currently enrolled students, and all other transcripts were sent to the 
school district. In this situation, the school would sometimes forward the request 
packet to the district. Other times, the school returned the materials to us, and we 
sent them to the school district. 

• The student transferred to another school. When the student had transferred to 
another school, a transcript was requested from the school to which the student had 
transferred. In some cases, the registrar or school secretary forwarded the request 
materials to the transfer school. In other cases, the request materials were sent back 
to us and we sent a new request to the transfer school. 

• The school would not release any transcript without student’s written consent. A 
few schools returned the transcript request materials with no transcripts, indicating 
that they required written consent from each student whose transcript we were 
requesting. A problem sheet was completed for these cases, and they were 
forwarded to the survey manager for follow-up. As a first step, the survey manager 
called the school to explain that, as an agent of ED, Mathematica was authorized to 
collect student transcripts for the purposes of this study and that, according to the 
laws of the Family Educational Rights and Privacy Act, schools are permitted to 
release student transcripts to ED without the written consent of students 
participating in the study. It was also explained that students had given verbal 
consent over the telephone or written consent when they completed the mail survey 
and that we did not request transcripts for any students who refused consent. Some 
schools agreed to send the requested transcripts on hearing this explanation. Others 
reiterated that signed consent was required by school policy. In this case, we sent 
written consent forms to the students for them to sign and return to Mathematica so 
that we could obtain their student transcript for the impact study. A postage-paid 
return envelope was included with the consent form. A small number of students 
did sign and return the consent form, but most of the letters came back unopened 
because we no longer had a valid address for the student. 

• The school would not release transcripts without advance payment. In these 
cases, we sent a check to cover the cost of each transcript, along with a list of the 
students whose transcripts we were requesting. 



3. For example, some school principals and registrars indicated that their school was often confused with another 
school having the same or a similar name and suggested that we direct our request to the other school. In this case, 
we would call the alternate school to find out whether the student was ever enrolled there. If so, we made a 
correction to the database and sent a request to the newly identified school. 
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• The school would not release a transcript until the student paid an outstanding 
debt. In some cases we were eventually able to obtain these transcripts as students 
paid whatever bills they owed the school. When the debt remained unpaid, 
however, there was no way we could get the transcript. These cases were marked 
as unfilled requests. 



4. Response Rates 

From the samples used in the impact analysis described in Chapter III, 1,201 students 
reported having attended at least one postsecondary institution that could be matched to the 
Postsecondary Education Data System. For each of these students, we requested a transcript from 
each of the postsecondary institutions that he or she reported attending. We received 1,756 of the 
2,295 transcripts requested (77 percent). 



C. NATIONAL STUDENT CLEARINGHOUSE DATA 

The National Student Clearinghouse (NSC) is a repository of information on enrollment, 
loans and degrees awarded for postsecondary institutions that join the NSC as members. The 
NSC is a nonprofit organization that began in 1993 with support from ED to verify enrollment 
for student loan recipients. Currently, the NSC is an enrollment and degree verification resource 
for ED, colleges and universities, and employers. It is supported by fees paid by the member 
institutions as well as by fees for enrollment and degree searches. Because the NSC began 
operations only two years before most of the sample members in this study were graduating from 
high school (1995-1997), caution is needed in using the data from this period. The NSC Web 
site reports that it reached about 25 percent coverage for enrollment data in 1996, and that it did 
not begin to collect data on degrees awarded until sometime later. Coverage is more complete for 
four-year institutions than for two-year and less-than-two-year institutions. By 2009, the NSC 
reported coverage of about 10,000 institutions representing about 92 percent of enrollment."^ We 
submitted identifying information for all 1,675 eligible sample members from the treatment and 
control groups; however, as noted above, because of the early time frame, NSC data should be 
used with caution because it probably underreports postsecondary entrance and degree 
attainment in this period. 



D. FEDERAL STUDENT FINANCIAL AID DATA 

The Office of Postsecondary Education in ED provided consistent, comprehensive data on 
applications for federal financial aid. We were able to obtain files including information on all 
sample members who applied for aid for each year of the follow-up period. The data source for 
all applications for federal financial aid is the Free Application for Federal Student Aid. The 
main data items of interest were application for financial aid and receipt of a Pell grant. The 
primary approach for searching for someone in the Student Financial Aid (SEA) data is based on 



4. Figures are based on http://www.studentclearinghouse.org. 
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the person’s Social Security number. For cases with a missing or obviously invalid Social 
Security number, attempts were made to match using name and date of birth. During the years of 
interest, 1,400 (84 percent, unweighted) of our sample members applied for aid in at least one 
year, and 1,108 (66 percent, unweighted) received a Pell grant. 



E. COMBINING DATA 

Each of our three data sources — the follow-up surveys conducted for the evaluation, the 
NSC, and the federal SEA files — contain valuable information for measuring postsecondary 
educational outcomes. However, each also has some important limitations. The limitations of the 
surveys include nonresponse and, potentially, response error. Eor the fifth follow-up survey, 26 
percent of sample members did not respond at all. Although there is very little item nonresponse 
among respondents, some respondents might not have answered questions correctly, forgetting, 
for example, a brief period of college enrollment several years earlier. The main limitation of the 
NSC is undercoverage; that is, not all postsecondary institutions are in the NSC. Because a 
substantial fraction of postsecondary enrollees never receive financial support from a Pell grant, 
the main limitation of the SPA data is measurement error and, specifically, the fact that Pell grant 
receipt or nonreceipt is not equivalent to postsecondary enrollment or nonenrollment. 

In using these data sources and determining how to address effectively their relative 
limitations, several questions arise, including the following. How might we combine the data 
sources? How much of the available data do we use, recognizing that the different data sources 
do not cover exactly the same periods of time? How do we handle cases for which the data that 
are used do not provide definitive evidence about postsecondary enrollment status? Because 
none of these questions has a single, unambiguously correct answer, our approach has been to 
develop many different measures of outcomes that reflect different reasonable answers to the 
questions. 



I. Enrollment Measures 

We constructed measures using only the fifth follow-up survey (Measure 1), only the NSC 
(Measure 2), and only the SPA data (Measure 3). We used the Pell receipt indicator variable 
from the SPA data for the construction of measures of enrollment because it was an indication 
that a school actually received Pell grant money after a student’s enrollment. The indicator of 
application for financial aid, in contrast, is not necessarily indicative of subsequent enrollment. 
We also created measures using each possible combination of data, including NSC and SPA 
(Measure 4), SPA and survey (Measure 6), NSC and survey (Measure 7), and all three data 
sources (Measures 5 and 8). The last two measures differ only in the order in which the data 
were used: Measure 5 uses all data sources simultaneously while Measure 8 uses the survey data 
first. The final combination. Measure 9, augments all three current sources with data from the 
third and fourth follow-up surveys. 

Although the survey and SPA data available cover roughly the same reference periods, the 
NSC data are available for a period of time after the completion of the fifth follow-up survey and 
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the years for which financial aid records were available to us. One advantage of truncating the 
NSC data at the end of 2004 is that we have a similar reference period for all data sources; 
however, this approach also means discarding relevant data for sample members who may have 
taken longer to enroll in school. Therefore, for all measures using NSC data, we created versions 
using the full data through 2006 and truncated data through 2004. 

For each combination of data sources that we consider, we ascertain the enrollment status of 
as many cases as we can with the information that is available in the applicable data sources. 
With Measure 5, for example, we code a sample member as an enrollee if he or she is found to 
be an enrollee in the NSC data or if he or she is a Pell recipient according to the SFA data or if 
the sample member said in the survey that he or she was enrolled at some time. The sample 
member is not an enrollee if he or she does not appear in the NSC data (and is therefore not an 
enrollee) and has not been a Pell recipient and said in the survey that he or she had never been 
enrolled. This approach leaves uncoded the survey nonrespondents who are not in the NSC data 
and did not receive a Pell grant. For them, we apply the following assumptions: (a) not enrolled, 
(b) not enrolled if never applied for financial aid (otherwise, left as enrollment status missing), 
and (c) left as enrollment status missing. With assumptions (b) and (c), cases left as enrollment 
status missing get dropped from the analysis, and weights for the remaining cases are adjusted to 
compensate, as described elsewhere in this appendix. When creating measures based on the 
truncated NSC data, we follow the same steps described above, but ignore all NSC records that 
have a starting date after July 31, 2004. 

Measures 4, 6 and 7 are constructed in similar fashion. Measure 4 combines the NSC and 
SFA data and has versions based on whether to use the full or truncated NSC file. Measure 6 
combines the fifth follow-up survey with the SFA data and has variations based on the 
assumptions applied to survey nonrespondents. Measure 7 combines the fifth follow-up survey 
with the NSC data and includes variations based on both the amount of NSC data to include and 
how to code nonrespondents; it does not, however, have assumption (b) for dealing with uncoded 
data because that assumption relies on SFA data. 

Measures 8 and 9 are similar to Measure 5, with some small exceptions. Rather than treat all 
data equivalently, as in Measure 5, Measure 8 establishes preference for the fifth follow-up 
survey. Variables are defined first using survey responses, and data from the other two sources 
are used only for survey nonrespondents. This approach implies that if we have inconsistent data 
from multiple sources, the survey data take precedence. Measure 9 uses Measure 5 as a starting 
point, and augments it with data from the third and fourth follow-up surveys. Any remaining 
uncoded observations use the following modified assumptions: (e) enrolled if reported 
enrollment in either the fourth or third follow-up surveys, otherwise not enrolled; (f) not enrolled 
if reported lack of enrollment in either the fourth or third follow-up surveys and if never applied 
for financial aid, otherwise left as enrollment status missing; and (g) left as enrollment status 
missing. 
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2. Completion Measures 



Just as the NSC data were unable to provide information on financial aid application and 
receipt, the SFA data were unable to provide information on postsecondary completion. In the 
creation of measures of completion, we are therefore limited to those that can be constructed 
using only the fifth follow-up survey and NSC data: Measure 1 (fifth follow-up survey only). 
Measure 2 (NSC only), and Measure 7 (the two sources combined). Furthermore, the degree 
completion information in the NSC is less comprehensive than enrollment information (just more 
than half of the schools provide degree information), adding to the limitations of the data for our 
analysis. 
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APPENDIX B 

PROGRAM IMPACTS BY SUBGROUPS 




EXHIBIT B.I. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON ANY POSTSECONDARY 
ENROLLMENT, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND (ITT) 



Table B.I 

Impact of UBMS on Any Postsecondary Enrollment by Prior Participation in Regular Upward Bound (ITT) 



Participated in Regular Upward Bound Did Not Participate in Regular Upward Bount 



Variable 


Data Source 


Uncoded 


UBMS 

Participants" 


Comparison 

Group" 


Impact*’ 




UBMS 

Participants" 


Comparison 

Q'oup" 


Impact*’ 


























1 


5th Follow-Up Survey ( Survey) 


Set to Missing Value 


99.45 


92.77 


6.67 


** 


97.38 


90.88 


6.49 




2 


NSC through 05-06 (NSCF) 


None 


100.00 


100.00 


1 




99.70 


100.00 


-0.30 




2T 


NSC through 03-04 (NSa) 


None 


97.83 


95.52 


2.30 




95.19 


96.00 


-0.81 




3 


Pell Receipt (Student Financial Aid - SFA) 


None 


74.87 


70.41 


4.46 




62.46 


67.43 


-4.98 




4 


NSCF /SFA 


None 


100.00 


100.00 


1 




99.83 


100.00 


-0.17 




4T 


NSa/SFA 


None 


98.85 


98.74 


0.11 




97.55 


98.97 


-1.42 




5A 


Survey /NSCF /SFA 


Set to 0 


100.00 


100.00 


] 




99.83 


100.00 


-0.17 




SAT 


Survey /NSa/ SFA 


Set to 0 


99.97 


98.79 


1.18 




93.66 


99.89 


-6.22 




5B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


100.00 


100.00 


1 




100.00 


100.00 


I 




5BT 


Survey /NSa/ SFA 


Set to 0 if no aid applied 


99.97 


98.79 


1.18 




96.99 


99.89 


-2.90 




5C 


Survey /NSCF /SFA 


Set to Missing Value 


100.00 


100.00 


1 




100.00 


100.00 


1 




5a 


Survey /NSa/ SFA 


Set to Missing Value 


99.86 


99.17 


0.68 




98.31 


99.92 


-1.61 




6A 


Survey / SFA 


Set to 0 


94.40 


88.48 


5.92 




90.21 


85.58 


4.63 




6B 


Survey / SFA 


Set to 0 if no aid applied 


98.03 


90.62 


7.41 


** 


95.11 


88.70 


6.40 


* 


6C 


Survey / SFA 


Set to Missing Value 


100.00 


93.95 


6.82 


:is 


97.29 


93.13 


4.17 


** 


7A 


Survey / NSCF 


Set to 0 


100.00 


100.00 


1 




99.70 


100.00 


-0.30 




VAT 


Survey /NSa 


Set to 0 


99.97 


98.79 


1.18 


*** 


98.82 


99.62 


-0.80 




7C 


Survey / NSCF 


Set to Missing Value 


100.00 


100.00 


1 




100.00 


100.00 


1 




va 


Survey /NSa 


Set to Missing Value 


99.86 


99.17 


0.68 




99.57 


99.86 


-0.29 




8 


Survey then NSCF / SFA 


Set to 0 


99.76 


93.92 


5.84 




97.58 


92.77 


4.81 




8T 


Survey then NSa / SFA 


Set to 0 


99.71 


93.53 


6.18 




97.41 


92.73 


4.68 




9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


100.00 


100.00 


1 




100.00 


100.00 


1 




9AT 


Mult Surveys (3rd-5th) / NSa / SFA 


Set to 0 


99.97 


98.79 


1.18 




96.99 


99.89 


-2.90 




9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid applied 


100.00 


100.00 


1 




100.00 


100.00 


1 




9BT 


Mult Surveys (3rd-5th) / NSa / SFA 


Set to 0 if no aid app 


99.97 


98.79 


1.18 


:is 


96.99 


99.89 


-2.90 




9C 


Mult Surveys (3rd-5th) / NSa / SFA 


Set to Missing Value 


100.00 


100.00 


1 




100.00 


100.00 


1 




9Q 


Mult Surveys (3rd-5th) / NSa / SFA 


Set to Missing Value 


99.97 


98.79 


1.18 




96.99 


99.89 


-2.90 





Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 



Note: ITT = Intent To Treat; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse 
truncated; SFA = Student Financ- 3 -ial Aid. All estimates are weighted to account for missing data. 

Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. The symbol ! indicates that impacts could not be assessed due to complete or semi-complete 
separation. 

All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

*/**/*** Statistically significant at the 0.1/0.05/0.01 levels, respectively. 
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EXHIBIT B.2. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON HIGHEST LEVEL OF POSTSECONDARY 
ENROLLMENT, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND: FOUR- YEAR INSTITUTION (ITT) 





Participated 


in Regular Upward Bound 




Did Not Participate in Regular Upward Bound 


Field of Study 


UBMS 

Participants^ 


Comparison 

Gtoup^ 


b 

Inpact 






UBMS 

Participants^ 


Comparison 

Q'oup^ 


b 

Impact 




































Field of Stu^ at Most Recent Postsecondary bistitution 




















All Postsecondary Institutions 




















Math and science fields, excluding social sciences 


39.26 


29.93 


9.34 






30.33 


24.81 


5.52 




Social science fields 


15.70 


7.70 


8.01 


* 




11.96 


8.36 


3.60 




Other fields 


36.77 


53.65 


-16.88 


** 




49.02 


58.08 


-9.06 




Four-year colleges and universities 




















Math and science fields, excluding social sciences 


32.76 


25.57 


7.19 






23.18 


19.03 


4.15 




Social science fields 


14.52 


6.98 


7.54 


* 




11.95 


7.45 


4.50 




Other fields 


30.20 


38.91 


-8.71 






45.22 


44.24 


0.98 




Enrned Degree or Certificate in Field 




















All Postsecondary Institutions 




















Math and science fields, excluding social sciences 


15.15 


14.14 


1.01 






15.90 


12.19 


3.71 




Social science fields 


7.71 


3.78 


3.93 






8.64 


4.56 


4.08 


* 


Other fields 


17.52 


27.85 


-10.33 






31.24 


31.95 


-0.71 




Four-year colleges and universities 




















Math and science fields, excluding social sciences 


9.10 


10.48 


-1.38 






10.85 


5.43 


5.42 




Social science fields 


5.45 


3.44 


2.01 






8.71 


4.10 


4.62 


** 


Other fields 


9.62 


15.23 


-5.60 






22.97 


23.54 


-0.57 





Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 



Note: ITT = Intent To Treat; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse 
truncated; SEA = Student Financial Aid. All estimates are weighted to account for missing data. 

“ Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

’’ All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

*/**/*** Statistically significant at the 0.1/0.05/0.01 levels, respectively. 
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EXHIBIT B.3. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON HIGHEST LEVEL OF POSTSECONDARY 
ENROLLMENT, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND: TWO-YEAR INSTITUTION (ITT) 



Table B.3 

Impact of Upward Bound Math Science on Highest Level of Postsecondary Enrollment by Prior Participation in Regular Upward Bound; Two-Year Institution (ITT) 



Participated in Regular Upward Bound Did Not Participate in Regular Upward Bound 







UBMS 


Comparison 

a Impact 

Group 


UBMS 


Comparison 


Data Source 


Uncoded 


Participants'* 


Participants* 


Group* 



1 


Sth Follow-Up Survey (Survey) 


Set to Missing Value 


12.15 


16.46 -4.31 


6.78 


10.39 -3.61 


2 


NSC through OS-06 (NSCF) 


None 


11.48 


1V.88 -6.40 


5.53 


15.49 -9.96 


2T 


NSC through 03-04 (NSCT) 


None 


12.68 


1V.51 -4.83 


5.01 


15.59 -10.58 


3 


Pell Receipt (Student Finacial Aid - SFA) 


None 


V.04 


6.82 0.23 


2.29 


9.12 -6.82 «* 


4 


NSCF/SFA 


None 


8.59 


V.47 1.13 


2.91 


10.97 -8.06 *** 


4T 


NSCT /SFA 


None 


8.V3 


7.27 1.47 


3.27 


10.97 -7.70 *** 


5A 


Survey /NSCF /SFA 


Set to 0 


8.V5 


7.87 0.88 


3.56 


10.80 -7.24 «* 


SAT 


Survey /NSa/ SFA 


Set to 0 


8.8V 


7.71 1.16 


3.93 


10.80 -6.87 *** 


SB 


Survey /NSCF /SFA 


Set to Oifno aid applied 


8.V5 


7.87 0.88 


3.56 


10.80 -7.23 


SBT 


Survey /NSCT /SFA 


Set to Oifno aid applied 


8.8V 


7.71 1.16 


3.78 


10.80 -7.01 *** 


SC 


Survey /NSCF /SFA 


Set to Missing Value 


8.V5 


7.87 0.88 


3.56 


10.80 -7.23 *** 


scr 


Survey /NSCT /SFA 


Set to Missing Value 


8.8V 


7.74 1.13 


3.79 


10.80 -7.01 *** 


6A 


Survey / SFA 


Set to 0 


8.V9 


7.36 1.43 


4.02 


9.73 -5.71 


6B 


Survey / SFA 


Set to Oifno aid applied 


9.08 


7.24 1.84 


3.75 


9.43 -5.67 


6C 


Survey / SFA 


Set to Missing Value 


9.26 


7.51 1.75 


3.83 


9.90 -6.07 


7A 


Survey / NSCF 


Set to 0 


8.39 


15.36 -6.98 


5.63 


13.34 -7.71 


VAT 


Survey / NSCT 


Set to 0 


10.10 


17.59 -7.49 


5.66 


13.25 -7.59 *** 


VC 


Survey / NSCF 


Set to Missing Value 


8.39 


15.36 -6.98 


5.66 


13.34 -7.68 *** 


7CT 


Survey / NSCT 


Set to Missing Value 


10.12 


17.66 -7.54 


5.70 


13.28 -7.58 *** 


8 


Survey then NSCF / SFA 


Set to 0 


8.V5 


7.87 0.88 


3.56 


10.80 -7.24 


8T 


Survey then NSCT / SFA 


Set to 0 


8.8V 


7.71 1.16 


3.93 


10.80 -6.87 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


8.83 


8.08 0.75 


3.36 


11.17 -7.81 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


8.95 


7.92 1.03 


3.73 


11.17 -7.44 


9B 


Mult Surveys (3rd -5th) / NSCF / SFA 


Set to Oifno aid applied 


8.83 


8.08 0.75 


3.36 


11.17 -7.81 *** 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Oifno aid applied 


8.95 


7.92 1.03 


3.73 


11.17 -7.44 


9C 


Mult Surveys (3rd -5th) / NSCF / SFA 


Set to Missing Value 


8.83 


8.08 0.75 


3.36 


11.17 -7.81 


9Q 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


8.95 


7.92 1.03 


3.73 


11.17 -7.44 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: ITT = Intent To Treat; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse 
truncated; SEA = Student Financial Aid. All estimates are weighted to account for missing data. 

‘‘ Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

*** Statistically significant at the 0.01 level. 
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EXHIBIT B.4. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON HIGHEST LEVEL OF POSTSECONDARY 
ENROLLMENT, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND: OTHER INSTITUTION (ITT) 



Table B.4 


Impact of Upward Bound Math Science on Highest Level of Postsecondaiy Enrollment by Prior Participation in Regular Upward Bound: Other Institution (ITT) 












Participated in Regular Upward Bound 




Did Not Participate in Regular Upward Bound 




Data Source 


Uncoded 


UBMS 

Participants” 


Comparison 

Group” 


Impact'’ 




UBMS 

Participants” 


Comparison 

Group” 


T b 

inpact 
















1 5th Follow-Up Survey (Survey) 


Set to Missing Value 


0.13 


3.82 


-3.69 




1.23 


2.53 


-1.31 




2 


NSC through 05-06 (NSCF) 


None 


0.17 


0.28 


-0.11 




1.88 


1.00 


0.89 




2T 


NSC through 03-04 (NSCT) 


None 


0.17 


0.28 


-0.11 




4.83 


0.96 


3.86 


*** 


3 


Pel Receipt (Student Financial Aid - SFA) 


None 


0.40 


2.93 


-2.53 




1.96 


2.81 


-0.85 




4 


NSCF/ SFA 


None 


0.31 


2.74 


-2.44 




2.01 


2.63 


-0.62 




4T 


NSCT /SFA 


None 


0.31 


2.74 


-2.44 




2.12 


2.60 


-0.47 




5A 


Survey /NSCF/ SFA 


Set to 0 


0.24 


3.11 


-2.87 




1.91 


2.48 


-0.58 




5AT Survey/ NSCT /SFA 


Set to 0 


0.24 


3.11 


-2.87 




2.02 


2.45 


-0.43 




5B 


Survey /NSCF /SFA 


Set to Oifno aid applied 


0.24 


3.11 


-2.87 




1.91 


2.48 


-0.58 




5BT Survey /NSCT /SFA 


Set to Oifno aid applied 


0.24 


3.11 


-2.87 




2.03 


2.45 


-0.42 




5C 


Survey /NSCF/ SFA 


Set to Missing Value 


0.24 


3.11 


-2.87 




1.91 


2.48 


-0.58 




5CT Survey /NSCT /SFA 


Set to Missing Value 


0.25 


3.13 


-2.87 




2.03 


2.45 


-0.42 




6A 


Survey / SFA 


Set to 0 


0.34 


3.27 


-2.93 




1.77 


2.87 


-1.10 




6B 


Survey / SFA 


Set to Oifno aid applied 


0.25 


3.19 


-2.94 




1.67 


2.92 


-1.25 




6C 


Survey / SFA 


Set to Missing Value 


0.15 


3.31 


-3.16 




1.75 


3.07 


-1.32 




Ik 


Survey / NSCF 


Set to 0 


0.20 


3.08 


-2.88 




1.72 


2.64 


-0.92 




7AT Survey /NSCT 


Set to 0 


0.13 


3.16 


-3.03 




1.91 


2.61 


-0.69 




7C 


Survey /NSCF 


Set to Missing Value 


0.20 


3.08 


-2.88 




1.73 


2.64 


-0.91 




7CT Survey /NSCT 


Set to Missing Value 


0.14 


3.17 


-3.03 




1.92 


2.61 


-0.69 




8 


Survey then NSCF /SFA 


Set to 0 


0.24 


3.11 


-2.87 




1.91 


2.48 


-0.58 




8T 


Survey then NSCT /SFA 


Set to 0 


0.24 


3.11 


-2.87 




2.02 


2.45 


-0.43 




9A 


Mult Surveys (3rd-5th)/ NSCF /SFA 


Set to 0 


0.25 


3.14 


-2.89 




1.90 


2.56 


-0.66 




9AT Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


0.25 


3.14 


-2.89 




2.01 


2.53 


-0.51 




9B 


Mult Surveys (3rd-5th)/ NSCF /SFA 


Set to Oifno aid applied 


0.25 


3.14 


-2.89 




1.90 


2.56 


-0.66 




9BT Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Oifno aid applied 


0.25 


3.14 


-2.89 




2.01 


2.53 


-0.51 




9C 


Mult Surveys (3rd-5th)/ NSCF /SFA 


Set to Missing Value 


0.25 


3.14 


-2.89 




1.90 


2.56 


-0.66 




9CT Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


0.25 


3.14 


-2.89 




2.01 


2.53 


-0.51 







Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: ITT = Intent To Treat; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse 
truncated; SEA = Student Financial Aid. All estimates are weighted to account for missing data. 

Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

*** Statistically significant at the 0.01 level. 
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EXHIBIT B.5. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON ATTENDING A HIGHLY SELECTIVE 
FOUR- YEAR POSTSECONDARY INSTITUTION, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND (ITT) 



Table B.5 


















Participated i 


in Regular Upward Bound 


Did Not Participate in Regular Upward Bount 


Variable 


Data Source 


Uncoded 


UBMS 

Participants^ 


Comparison 

Group^ 


. b 

Impact 




UBMS 

Participants^ 


Comparison 
Group ^ 


. b 

Impact 
































1 


Sth Follow-Up Survey (Survey) 


Set to Missing Value 


35.04 


27.63 


7.41 




42.91 


26.96 


15.95 


*** 


2 


NSC through OS-06 (NSCF) 


None 


30.08 


19.59 


10.48 




37.96 


15.53 


22.43 


*** 


2T 


NSC through 03-04 (NSQ) 


None 


29.27 


18.48 


10.80 




36.68 


15.32 


21.37 


*** 


3 


Pell Receipt (Student Financial Aid - SFA) 


None 


55.72 


37.68 


18.04 




55.02 


31.64 


23.38 


*** 


4 


NSCF/SFA 


None 


57.66 


39.55 


18.11 




56.54 


32.41 


24.13 


*** 


4T 


NSCT /SFA 


None 


57.25 


38.84 


18.41 




55.96 


32.29 


23.67 


*** 


5A 


Survey /NSCF /SFA 


Set to 0 


61.02 


42.22 


18.80 




56.78 


36.19 


20.59 


*** 


SAT 


Survey /NSCT /SFA 


Set to 0 


60.62 


41.51 


19.11 




56.20 


36.07 


20.13 


*** 


SB 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


61.02 


42.22 


18.80 


*** 


56.88 


36.19 


20.69 


*** 


SBT 


Survey /NSCT /SFA 


Set to 0 if no aid applied 


60.62 


41.51 


19.11 




56.37 


36.07 


20.30 


*** 


SC 


Survey /NSCF /SFA 


Set to Missing Value 


61.02 


42.22 


18.80 




56.88 


36.19 


20.69 


*** 


sa 


Survey /NSCT /SFA 


Set to Missing Value 


60.65 


41.68 


18.98 




56.38 


36.08 


20.30 


*** 


6A 


Survey / SFA 


Set to 0 


59.33 


40.58 


18.75 




56.04 


36.07 


19.97 


*** 


6B 


Survey / SFA 


Set to Oifno aid applied 


59.50 


40.27 


19.23 




56.26 


36.25 


20.01 


*** 


6C 


Survey / SFA 


Set to Missing Value 


60.30 


41.75 


18.56 




59.74 


35.42 


24.32 


*** 


7A 


Survey /NSCF 


Set to 0 


35.85 


28.66 


7.19 




40.52 


26.23 


14.29 




7AT 


Survey /NSCT 


Set to 0 


35.07 


27.54 


7.53 




39.70 


26.11 


13.58 




7C 


Survey /NSCF 


Set to Missing Value 


35.85 


28.66 


7.19 




40.66 


26.23 


14.43 




7Q 


Survey /NSCT 


Set to Missing Value 


35.12 


27.65 


7.47 




40.08 


26.17 


13.91 




8 


Survey then NSCF/ SFA 


Set to 0 


61.02 


42.22 


18.80 




56.78 


36.19 


20.59 


*** 


8T 


Survey then NSCT / SFA 


Set to 0 


60.62 


41.51 


19.11 




56.20 


36.07 


20.13 


*** 















Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 



Note: ITT = Intent To Treat; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse 
truncated; SEA = Student Financial Aid. All estimates are weighted to account for missing data. 

Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

*/**/*** Statistically significant at the 0.1/0.05/0.01 levels, respectively. 
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EXHIBIT B.6. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON TOTAL POSTSECONDARY 
CREDITS EARNED, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND (ITT) 





Participated 


1 in Regular Upward Bound 




Did Not Participate in Regular Upward Bound 


Postsecondary Credits Earned 


UBMS 

Participants* 


Comparison 

Croup* 


Impact*’ 






UBMS 

Participants* 


Comparison 

Croup* 


Impact*’ 


















Two- and four-year college and university 


100.77 


S5.93 


14.84 






93.65 


83.35 


10.31 




Two- year colleges 


15.31 


14.56 


0.76 






8.00 


10.96 


-2.96 




Four-year colleges 


71.00 


55.81 


15.19 






85.89 


70.84 


15.06 




Math and Science 


33.92 


24.33 


9.60 






28.43 


21.01 


7.42 


* 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: ITT = Intent To Treat. All estimates are weighted to account for missing data. 

‘ Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

* Statistically significant at the 0. 1 level. 
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EXHIBIT B.7. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON COMPLETION OF ANY CREDENTIAL AND 
HIGHEST CREDENTIAL COMPLETED, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND (ITT) 



Table B.7 



Impact of Upward Bound Math Science on Completed Any Credential and Highest Credential Completed by Prior Participation in Regular Upward Bound (ITT) 









Participatec 


1 in Regular Upward Bound 


Did Not Participate i 


in Regular Upward Bound 




Variable 


Outcome / Data Source 


Variable 


Uncoded 


UBMS 

Participants 


Conparison 

Qoup^ 




UBMS 

Participants^ 


Conparison 

&oup^ 


T b 

Impact 
























Any Postsecondary Degree 


Any Postsecondary Degree 








1 


Survey 


1 


Set to Missing Value 


56.54 


58.84 




72.86 


61.25 


11.60 


* 


2 


NSCF 


2 


None 


44.75 


34.99 




47.89 


38.07 


9.82 


** 


2T 


NSCT 


2T 


None 


40.49 


30.53 




44.86 


36.37 


8.49 


* 


7A 


Survey / NSCF 


7A 


Set to 0 


59.32 


53.07 




67.34 


54.37 


12.97 




7AT 


Survey / NSCT 


7AT 


Set to 0 


55.51 


50.50 




65.39 


53.33 


12.05 


*** 


7B 


Survey /NSCF /SFA 


7B 


Set toOifno aid applied 


59.32 


53.07 




67.53 


54.37 


13.16 


*** 


7BT 


Survey /NSa/ SFA 


7BT 


Set toOifno aid applied 


55.51 


50.50 




65.84 


53.44 


12.40 




1C 


Survey / NSCF 


7C 


Set to Missing Value 


59.32 


53.07 




67.53 


54.37 


13.16 


*** 


ICY 


Survey / NSCT 


7CT 


Set to Missing Value 


55.59 


50.70 




65.86 


53.46 


12.40 




Highest Degree Completed: Four-Year Degree 


Highest Degree Conpleted: Four- Year Degree 
















1 


Survey 


1 


Set to Missing Value 


44.05 


41.59 




64.46 


45.83 


18.63 


** 


2 


NSCF 


2 


None 


42.14 


28.65 


* 


46.68 


32.28 


14.40 




2T 


NSCT 


2T 


None 


36.19 


25.38 




43.34 


31.37 


11.97 


** 


7A 


Survey / NSCF 


7A 


Set to 0 


50.92 


38.94 




59.93 


42.39 


17.54 




7AT 


Survey / NSCT 


7AT 


Set to 0 


45.56 


36.96 




57.98 


41.48 


16.49 




7B 


Survey /NSCF /SFA 


7B 


Set toOifno aid applied 


50.92 


38.94 




60.09 


42.39 


17.70 


*** 


7BT 


Survey /NSCT /SFA 


7BT 


Set toOifno aid applied 


45.56 


36.96 




58.36 


41.57 


16.79 




1C 


Survey / NSCF 


7C 


Set to Missing Value 


50.92 


38.94 




60.09 


42.39 


17.70 


*** 


ICY 


Survey / NSCT 


7CT 


Set to Missing Value 


45.61 


37.11 




58.37 


41.58 


16.79 


*** 


Highest Degree Completed: Two-Year Degree 


Highest Degree Conpleted: Two-Year Degree 
















1 


Survey 


1 


Set to Missing Value 


9.90 


10.28 




5.09 


11.49 


-6.40 


** 


2 


NSCF 


2 


None 


1.15 


5.71 


* 


1.55 


5.62 


4.07 




2T 


NSCT 


2T 


None 


4.88 


4.52 




1.70 


4.95 


-3.25 


** 


7A 


Survey / NSCF 


7A 


Set to 0 


6.28 


9.20 




4.78 


9.22 


4.45 


** 


7AT 


Survey / NSCT 


7AT 


Set to 0 


8.83 


8.52 




4.79 


8.99 


4.20 


* 


7B 


Survey /NSCF /SFA 


7B 


Set toOifno aid applied 


6.28 


9.20 




4.80 


9.22 


4.43 


** 


7BT 


Survey /NSCT /SFA 


7BT 


Set toOifno aidapphed 


8.83 


8.52 




4.84 


9.01 


4.17 


* 


1C 


Survey / NSCF 


7C 


Set to Missing Value 


6.28 


9.20 




4.80 


9.22 


4.43 


** 


ICY 


Survey / NSCT 


7CT 


Set to Missing Value 


8.85 


8.55 




4.84 


9.01 


-4.17 


* 


Highest Degree Completed: Other Degree 


Highest Degree Completed: Other Degree 
















1 


Survey 


Set to Missing Value 


3.42 


6.96 


-3.54 




4.06 


3.94 


0.12 




2 


NSCF 


None 


-0.06 


0.63 


-0.69 




0.56 


0.17 


0.39 




2T 


NSCT 


None 


-0.06 


0.63 


-0.69 




0.33 


0.04 


0.28 


* 


7A 


Survey / NSCF 


Set to 0 


2.53 


4.93 


-2.40 




3.33 


2.76 


0.58 




7AT 


Survey / NSCT 


Set to 0 


2.54 


5.02 


-2.48 




3.36 


2.86 


0.50 




7B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


2.53 


4.93 


-2.40 




3.34 


2.76 


0.58 




7BT 


Survey /NSCT /SFA 


Set to 0 if no aid applied 


2.54 


5.02 


-2.48 




3.37 


2.87 


0.51 




1C 


Survey / NSCF 


Set to Missing Value 


2.53 


4.93 


-2.40 




3.34 


2.76 


0.58 




ICY 


Survey / NSCT 


Set to Missing Value 


2.54 


5.04 


-2.50 




3.37 


2.87 


0.51 





Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: ITT = Intent To Treat; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse 
truncated; SFA = Student Financial Aid. All estimates are weighted to account for missing data. 

Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

’’ All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 

*/**/*** Statistically significant at the 0.1/0.05/0.01 levels, respectively. 
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EXHIBIT B.8. IMPACT OF UPWARD BOUND MATH-SCIENCE (UBMS) ON POSTSECONDARY FIELD OF 
STUDY, BY PRIOR PARTICIPATION IN REGULAR UPWARD BOUND (ITT) 





Participated 


in Regular Upward Bound 




Did Not Participate in Regular Upward Bound 


Field of Study 


UBMS 

Participants^ 


Conparison 

Gtoup^ 


T b 

Inpact 






UBMS 

Participants^ 


Comparison 

Gtoup^ 


T b 

Inpact 
























Field of Stu(fy at Most Recent Postsecondary bistitution 












AU Postsecondary Institutions 












Math and science fields, excluding social sciences 


39.26 


29.93 


9.34 




30.33 


24.81 


5.52 




Social science fields 


15.70 


7.70 


8.01 


* 


11.96 


8.36 


3.60 




Other fields 


36.77 


53.65 


-16.88 


=}C5fC 


49.02 


58.08 


-9.06 




Four-year colleges and universities 














Math and science fields, excluding social sciences 


32.76 


25.57 


7.19 






23.18 


19.03 


4.15 




Social science fields 


14.52 


6.98 


7.54 


* 




11.95 


7.45 


4.50 




Other fields 


30.20 


38.91 


-8.71 






45.22 


44.24 


0.98 




Earned Degree or Certificate in Field 




















AUPostsecondaiy Institutions 




















Math and science fields, excluding social sciences 


15.15 


14.14 


1.01 






15.90 


12.19 


3.71 




Social science fields 


7.71 


3.78 


3.93 






8.64 


4.56 


4.08 


* 


Other fields 


17.52 


27.85 


-10.33 






31.24 


31.95 


-0.71 




Four-year colleges and universities 




















Math and science fields, excluding social sciences 


9.10 


10.48 


-1.38 






10.85 


5.43 


5.42 




Social science fields 


5.45 


3.44 


2.01 






8.71 


4.10 


4.62 


** 


Other fields 


9.62 


15.23 


-5.60 






22.97 


23.54 


-0.57 





Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 



Note: ITT = Intent To Treat. All estimates are weighted to account for missing data. 

Impact may not exactly equal the difference between UBMS Participants and the Matched Comparison Group 
due to rounding. 

All estimates are weighted to account for missing data. The comparison group estimates and impact estimates 
are regression adjusted (see Chapter III, Section A.4). 



*/** Statistically significant at the 0.1/0.05 levels, respectively. 
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APPENDIX C 



SAMPLE SIZES AND STANDARD ERRORS 




EXHIBIT C.I. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT m.3 



Tabled 

Sample Sizes and Standard Errors for Reported Impact Estimates 



Variable 


Data Source 


Uncoded 


Sample Size 




Standard Error 
















1 


Sth Follow-Up Survey (Survey) 


Set to Missing Value 


1,338 


1.028 


2 


NSC through OS-06 (NSCF) 


None 


1,677 




0.108 


2T 


NSC through 03-04 (NSCT) 


None 


1,677 




1.333 


3 


PeU Receipt (Student Financial Aid -SFA) 


None 


1,677 




4.326 


4 


NSCF /SFA 


None 


1,677 




0.066 


4T 


NSCT / SFA 


None 


1,677 




1.949 


5A 


Survey /NSCF /SFA 


Set to 0 


1,677 




0.066 


SAT 


Survey / NSCT / SFA 


Set to 0 


1,677 




0.638 


SB 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


1,675 




!'’ 


SBT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,675 




0.461 


SC 


Survey /NSCF /SFA 


Set to Missing Value 


1,672 




!'’ 


SCT 


Survey / NSCT / SFA 


Set to Missing Value 


1,672 




0.619 


6A 


Survey / SFA 


Set to 0 


1,677 


2.769 


6B 


Survey / SFA 


Set to 0 if no aid applied 


1,611 




1.958 


6C 


Survey / SFA 


Set to Missing Value 


1,542 




0.922 


7A 


Survey / NSCF 


Set to 0 


1,677 




0.108 


7AT 


Survey / NSCT 


Set to 0 


1,677 




0.669 


7C 


Survey / NSCF 


Set to Missing Value 


1,666 




!'’ 


7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




0.448 


8 


Survey then NSCF / SFA 


Set to 0 


1,677 


0.894 


8T 


Survey then NSCT / SFA 


Set to 0 


1,677 




0.908 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


1,677 






9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


1,677 


0.461 


9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid applied 


1,677 






9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid app 


1677 




0.461 


9C 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


1677 


! 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


1677 


0.461 



Source; Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = 
Student Financial Aid. Statistical significance could not be assessed due to complete or semi-complete 
separation. 

“ Standard errors account for project clustering and were estimated using Taylor series linearization 
methods. 

’’ The symbol ! indicates that impacts could not be assessed due to complete or semi-complete separation. 
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EXHIBIT C.2. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT 

ESTIMATES IN EXHIBIT III.4 



Variable 


Data Source 


Uncoded 


Sample Size 




Standard Errorf 






















1 


5th Follow-Up Survey (Survey) 


Set to Missing Value 


1,338 




2.465 


2 


NSC through 05-06 (NSCF) 


None 


1,677 




3.821 


2T 


NSC through 03-04 (NSCT) 


None 


1,677 




3.822 


3 


Pell Receipt (SFA) 


None 


1,677 




3.038 


4 


NSCF/ SFA 


None 


1,677 




2.687 


4T 


NSCT / SFA 


None 


1,677 




2.781 


5A 


Survey / NSCF / SFA 


Set to 0 


1,677 




2.434 


5AT 


Survey /NSCT /SFA 


Set to 0 


1,677 




2.510 


5B 


Survey / NSCF / SFA 


Set to 0 if no aid applied 


1,676 




2.434 


5BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,675 




2.498 


5C 


Survey / NSCF / SFA 


Set to Missing Value 


1,676 




2.434 


5CT 


Survey / NSCT / SFA 


Set to Missing Value 


1,672 




2.497 


6A 


Survey / SFA 


Set to 0 


1,677 




2.634 


6B 


Survey / SFA 


Set to 0 if no aid applied 


1,611 




2.653 


6C 


Survey / SFA 


Set to Missing Value 


1,542 




1.949 


7A 


Survey / NSCF 


Set to 0 


1,677 




2.901 


7AT 


Survey / NSCT 


Set to 0 


1,677 




2.935 


1C 


Survey / NSCF 


Set to Missing Value 


1,675 




2.885 


7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




2.883 


8 


Survey then NSCF / SFA 


Set to 0 


1,677 




2.434 


8T 


Survey then NSCT / SFA 


Set to 0 


1,677 




2.510 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


1,677 




2.147 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


1,677 




2.214 


9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid applied 


1,677 




2.147 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid applied 


1,677 




2.214 


9C 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


1,677 




2.147 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


1,677 




2.214 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas. 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods 
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EXHIBIT C.3. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT III.5 



Variable 


Data Source 


Uncoded 


N 


SanpleSize 


1 


Sth R)llow-Up Survey (SurvQ^) 


Set to Mssing ValiE 


2102 


1,338 


2 


NSC through QS-06(r«CF) 


Ncaie 


im 


1,677 


2T 


NSCtlrough03-(»fr«CT) 


Ncaie 


2844 


1,677 


3 


Pdl Receipt (Studait Financial Aid - SFA) 


Ncme 


2844 


1,677 


4 


NSCF/SFA 


Ncaie 


2844 


1,677 


4T 


NSCT/ SFA 


Ncaie 


2844 


1,677 


5A 


SurvQf / / SFA 


Set to 0 


2m 


1,677 


SAT 


Survey / ISKCT / SFA 


Set to 0 


2m 


1,677 


SB 


SurvQf / ISKCF / SFA 


Set to 0 if no aid applied 


mi 


1,676 


SBT 


Survey/f«Cr/SFA 


Set to 0 if no aid applied 


2780 


1,675 


SC 


SurvQf / bBCF / SFA 


Set to Mssing Valie 


2581 


1,676 


scr 


Survey/NSCT/SFA 


Set to Mssing Valie 


2567 


1,672 


6A 


Surv^ / SFA 


Set to 0 


2844 


1,677 


6B 


SurvQf / SFA 


Set to 0 if no aid applied 


2717 


1,611 


6C 


Surv^ / SFA 


Set to Mssing ValiE 


2476 


1,54-2 


7A 


SurvQ'/bBQ=' 


Set to 0 


2844 


1,677 


TAT 


Survey /bBCT 


Set to 0 


2844 


1,677 


1C 


Surv^/bBO^ 


Set to Mssing ValiE 


2482 


1,675 


7CT 


Survey /bBCT 


Set to Mssing ValiE 


2457 


1,666 


8 


Surv^ then NSCF/ SA 


Set to 0 


2844 


1,677 


8T 


SurvQf then NSCT/ SFA 


&ttoO 


2844 


1,677 


9A 


Mjlt Survq« (3id-Sth) /NSCF/ SFA 


Set to 0 


2844 


1,677 


9AT 


Mjlt SurvQ« (3id-Sth) / NSCT / SA 


Set to 0 


2844 


1,677 


9B 


Mjlt Survq« (3id-Sth) /NSCF/ SFA 


Set to 0 if no aid applied 


2814 


1,677 


9BT 


Mjlt SurvQ« (3id-Sth) / NSCT / SA 


Set to 0 if no aid applied 


2809 


1,677 


9C 


Milt Survqjs (3id-Sth) /NSCF/ SFA 


Set to Mssing ValiE 


2748 


1,677 


9CT 


Milt SurvQ« (3id-Sth) / NSCT / SA 


Set to Mssing Valie 


2743 


1,677 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 



Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SEA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.4. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT III.6 



Table C.4 

Sample Sizes and Standard Errors for Reported Impact Estimates: Table IV.4 



Variable 


Data Source 


Uncoded 


Sample Size 




Standard Errorf 


























1 


Sth FoUow-Up Survey (Survey) 


Set to Missing Value 


1,338 




0.719 


2 


NSC through OS-06 (NSCF) 


None 


1,677 




0.808 


2T 


NSC through 03-04 (NSCT) 


None 


1,677 




1.048 


3 


PeU Receipt (SFA) 


None 


1,677 


0.796 


4 


NSCF / SFA 


None 


1,677 




0.835 


4T 


NSCT / SFA 


None 


1,677 


0.869 


5A 


Survey / NSCF / SFA 


Set to 0 


1,677 




0.841 


SAT 


Survey / NSCT / SFA 


Set to 0 


1,677 




0.873 


SB 


Survey / NSCF / SFA 


Set to 0 if no aid applied 


1,676 




0.841 


SBT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,675 




0.874 


SC 


Survey / NSCF / SFA 


Set to Missing Value 


1,676 




0.841 


SCT 


Survey / NSCT / SFA 


Set to Missing Value 


1,672 




0.874 


6A 


Survey / SFA 


Set to 0 


1,677 




0.759 


6B 


Survey / SFA 


Set to 0 if no aid applied 


1,611 




0.790 


6C 


Survey / SFA 


Set to Missing Value 


1,542 




0.817 


7A 


Survey / NSCF 


Set to 0 


1,677 


0.642 


7AT 


Survey / NSCT 


Set to 0 


1,677 


0.679 


7C 


Survey / NSCF 


Set to Missing Value 


1,675 


0.642 


7CT 


Survey / NSCT 


Set to Missing Value 


1,666 


0.683 


8 


Survey then NSCF / SFA 


Set to 0 


1,677 




0.841 


8T 


Survey then NSCT / SFA 


Set to 0 


1,677 




0.873 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


1,677 




0.828 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


1,677 




0.859 


9B 


Mult Surveys (3rd -Sth) / NSCF / SFA 


Set to 0 if no aid applied 


1,677 




0.828 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid applied 


1,677 




0.859 


9C 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


1,677 




0.828 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


1,677 




0.859 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.5. SAMPLE SIZES AND STANDARD ERRORS FOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT III.7 



Table C.5 

Sample Sizes and Standard Errors for Reported Impact Estimates 



Variable 


Data Source 


Uncoded 


Sample Size 


Standard Errorf 
























1 


5th Follow-Up Survey (Survey) 


Set to Missing Value 


1,338 




5.857 


2 


NSC through 05-06 (NSCF) 


None 


1,677 




6.402 


2T 


NSC through 03-04 (NSCT) 


None 


1,677 




6.390 


3 


PeU Receipt (Student Financial Aid - SFA) 


None 


1,677 




5.212 


4 


NSCF /SFA 


None 


1,677 




5.020 


4T 


NSCT / SFA 


None 


1,677 




5.072 


5A 


Survey /NSCF /SFA 


Set to 0 


1,677 




5.330 


SAT 


Survey / NSCT / SFA 


Set to 0 


1,677 




5.380 


5B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


1,676 




5.324 


5BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,675 




5.369 


5C 


Survey /NSCF /SFA 


Set to Missing Value 


1,676 




5.324 


5CT 


Survey / NSCT / SFA 


Set to Missing Value 


1,672 




5.366 


6A 


Survey / SFA 


Set to 0 


1,677 




5.423 


6B 


Survey / SFA 


Set to 0 if no aid applied 


1,611 




5.459 


6C 


Survey / SFA 


Set to Missing Value 


1,542 




5.418 


7A 


Survey / NSCF 


Set to 0 


1,677 




5.803 


7AT 


Survey / NSCT 


Set to 0 


1,677 




5.809 


7C 


Survey / NSCF 


Set to Missing Value 


1,675 




5.807 


7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




5.797 


8 


Survey then NSCF / SFA 


Set to 0 


1,677 




5.330 


8T 


Survey then NSCT / SFA 


Set to 0 


1,677 




5.380 



Source: Mathematica Policy Research, Inc., analysis file ‘TTT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = 
Student Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.6. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT III.8 



Postsecondary Credits Earned 


Sample Size 


Standard Error t 


Two- and four -year college and university 


1,677 


8.357 


T wo- year colleges 


1,296 


2.310 


Four-year colleges 


1,296 


7.064 


Math and Science 


1,677 


3.738 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.7. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT III.9 



Sample Sizes and Standard Errors for Reported Impact Estimates: Table IV.7 



Variable 


Outcome / Data Source 


Uncoded 


Sample Size 




Standard Error f 


























Any Postsecondary Degree 












1 


Survey 


Set to Missing Value 


1,201 




4.912 




2 


NSCF 


None 


1,677 




4.160 




2T 


NSCT 


None 


1,677 




4.318 




7A 


Survey / NSCF 


Set to 0 


1,677 




3.824 




7AT 


Survey / NSCT 


Set to 0 


1,677 




3.962 




7B 


Survey/ NSCF /SFA 


Set to 0 if no aid applied 


1,675 




3.823 




7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,669 




3.946 




7C 


Survey / NSCF 


Set to Missing Value 


1,675 




3.823 




7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




3.945 




Highest Degree Completed: Four-Year Degree 








1 


Survey 


Set to Missing Value 


1,201 




6.419 




2 


NSCF 


None 


1,677 




4.847 




2T 


NSCT 


None 


1,677 




4.944 




7A 


Survey / NSCF 


Set to 0 


1,677 




4.659 




7AT 


Survey / NSCT 


Set to 0 


1,677 




4.830 




7B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


1,675 




4.664 




7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,669 




4.825 




7C 


Survey / NSCF 


Set to Missing Value 


1,675 




4.664 




7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




4.822 




Highest Degree Completed: Two-Year Degree 






1 


Survey 


Set to Missing Value 


1,201 




1.782 




2 


NSCF 


None 


1,677 




0.746 




2T 


NSCT 


None 


1,677 




0.887 




7A 


Survey / NSCF 


Set to 0 


1,677 




1.366 




7AT 


Survey / NSCT 


Set to 0 


1,677 




1.495 




7B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


1,675 




1.368 




7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,669 




1.501 




7C 


Survey / NSCF 


Set to Missing Value 


1,675 




1.368 




7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




1.501 




Highest Degree Completed: Other Deg 


ree 








1 


Survey 


Set to Missing Value 


1,201 




1.325 


2 


NSCF 


None 


1,677 




0.453 


2T 


NSCT 


None 


1,677 




0.701 


7A 


Survey / NSCF 


Set to 0 


1,677 




1.006 


7AT 


Survey / NSCT 


Set to 0 


1,677 




1.014 


7B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


1,675 




1.008 


7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


1,669 




1.018 


7C 


Survey / NSCF 


Set to Missing Value 


1,675 




1.008 


7CT 


Survey / NSCT 


Set to Missing Value 


1,666 




1.018 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF=National Student Clearinghouse full; NSCT=National Student Clearinghouse truncated; SFA=Student Financial 
Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.8. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

EXHIBIT III. 10 



Table C.8 

S ample Sizes and Standard Errors for Reported Impact Estimates 



Field of Study 


S ample Size 


Standard E rrorf 


Field of Study at Most Recent Postsecondary Institution 

All Postsecondary Institutions 


Math and science fields, excluding social sciences 


1,271 


4.787 


Social science fields 


1,271 


3.430 


Other fields 


1,271 


4.961 


Four- year colleges and universities 


Math and science fields, excluding social sciences 


1,271 


4.592 


Social science fields 


1,271 


3.647 


Other fields 


1,271 


4.977 


Earned Degree or Certificate in Field 

All Postsecondary Institutions 


Math and science fields, excluding social sciences 


1,271 


3.462 


Social science fields 


1,271 


2.585 


Other fields 


1,271 


3.565 


Four- year colleges and universities 


Math and science fields, excluding social sciences 


1,271 


3.720 


Social science fields 


1,271 


2.716 


Other fields 


1,271 


3.716 



Source; Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 



t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.9. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.I 



Table C.9 



Sanple Sizes and Standard Errors for Reported Inpact Estimates: Table B.I 





















Participated in Regular Upward Bound 




Did Not Participate in Regular Upward Bound 


Variable 


Data Source 


Uncoded 


Sanple Size 


Standard EiTor^ 




Sanple Size 


Standard Error^ 


















1 5th Follow-Up Survey (Survey) 


Set to Missing Value 


510 


1.598 


828 


1.182 


2 


NSC through OS-06 (NSCF) 


None 


640 


!” 


1,037 


0.139 


2T 


NSC through 03-04 (NSCT) 


None 


640 


1.788 


1,037 


1.790 


3 


Pell Receipt (SEA) 


None 


640 


5.453 


1,037 


5.127 


4 


NSCF/SFA 


None 


640 


!” 




1,037 


0.101 


4T 


NSCT/SFA 


None 


640 


2.280 




1,037 


2.991 


5A 


Survey /NSCF/SFA 


Set to 0 


640 


!” 




1,037 


0.101 


SAT 


Survey /NSCT/SFA 


Set to 0 


640 


0.000 




1037 


0.749 


SB 


Survey /NSCF/SFA 


Set to 0 if no aid applied 


640 


!” 




1,035 


!” 


SBT 


Survey /NSCT/SFA 


Set to 0 if no aid applied 


640 


0.000 




1,035 


0.487 


SC 


Survey /NSCF/SFA 


Set to Missing Value 


638 


!” 




1,034 


!” 


SCT 


Survey /NSCT/SFA 


Set to Missing Value 


638 


0.001 




1,034 


0.320 


6A 


Survey / SFA 


Set to 0 


640 


2.789 


1,037 


3.297 


6B 


Survey / SFA 


Set to 0 if no aid applied 


617 


1.680 


994 


2.378 


6C 


Survey / SFA 


Set to Missing Value 


589 


0.000 


953 


1.115 


7A 


Survey / NSCF 


Set to 0 


640 


!” 




1,037 


0.139 


7AT 


Survey / NSCT 


Set to 0 


640 


0.000 




1,037 


1.104 


7C 


Survey / NSCF 


Set to Missing Value 


638 


!” 




1,028 


!” 


7CT 


Survey / NSCT 


Set to Missing Value 


638 


0.001 




1,028 


0.722 


8 


Survey then NSCF / SFA 


Set to 0 


640 


1.201 




1,037 


1.058 


8T 


Survey then NSCT / SFA 


Set to 0 


640 


1.190 




1,037 


1.081 


9A 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


640 


!“ 




1,037 


!" 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


640 


0.000 




1,037 


0.487 


9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid applied 


640 


!” 




1,037 


!” 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid applied 


640 


0.000 




1,037 


0.487 


9C 


Mult Sui-veys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


640 


!” 




1,037 


!” 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


640 


0.000 




1,037 


0.487 









Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SPA = Student Financial 
Aid. Statistical significance could not be assessed due to complete or semi-complete separation. 

Standard errors account for project clustering and were estimated using Taylor series linearization methods. 

’’ The symbol ! indicates that impacts could not be assessed due to complete or semi-complete separation. 
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EXHIBIT C.IO. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.2 





Participated in Regular Upward Bound 




Did Not Participate in 


Regular Upward Bound 


Variable 


Data Source 


Uncoded 


Sample Si 2 e 


Standard Errorf 




Sample Size 


Standard Errort 














1 5th Follow-Up Survey (Survey) 


Set to Missing Value 


510 


4.781 




828 


2.746 


2 


NSC through 05-06 (NSCF) 


None 


640 


5.910 




1,037 


4.242 


2T 


NSC through 03-04 (NSCT) 


None 


640 


6.132 




1,037 


4.231 


3 


A 


None 


640 


3.922 




1,037 


3.499 


4 


NSCF/ SFA 


None 


640 


3.754 




1,037 


3.030 


4T 


NSCT/SFA 


None 


640 


3.890 




1,037 


3.137 


5A 


Survey /NSCF/ SFA 


Set to 0 


640 


3.363 




1,037 


2.732 


5AT 


Survey /NSCT /SFA 


Set to 0 


640 


3.362 




1,037 


2.836 


5B 


Survey /NSCF/ SFA 


Set to 0 if no aid applied 


640 


3.363 




1,036 


2.733 


5BT 


Survey /NSCT /SFA 


Set to 0 if no aid applied 


640 


3.362 




1,035 


2.822 


5C 


Survey /NSCF/ SFA 


Set to Missing Value 


640 


3.363 




1,036 


2.733 


5CT 


Survey /NSCT /SFA 


Set to Missing Value 


638 


3.356 




1,034 


2.820 


6A 


Survey / SFA 


Set to 0 


640 


3.354 




1,037 


3.037 


6B 


Survey / SFA 


Set to 0 if no aid applied 


617 


3.497 




994 


3.027 


6C 


Survey / SFA 


Set to Missing Value 


589 


3.290 




953 


2.219 


7A 


Survey /NSCF 


Set to 0 


640 


4.439 




1,037 


3.285 


VAT 


Survey /NSCT 


Set to 0 


640 


4.575 




1,037 


3.321 


7C 


Survey /NSCF 


Set to Missing Value 


640 


4.439 




1,035 


3.257 


7cr 


Survey /NSCT 


Set to Missing Value 


638 


4.573 




1,028 


3.260 


8 


Survey then NSCF /SFA 


Set to 0 


640 


3.363 




1,037 


2.732 


8T 


Survey then NSCT /SFA 


Set to 0 


640 


3.362 




1,037 


2.836 


9A 


Mult Surveys (3rd-5th)/ NSCF / SFA 


Set to 0 


640 


3.277 




1,037 


2.394 


9AT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


640 


3.278 




1,037 


2.480 


9B 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid applied 


640 


3.277 




1,037 


2.394 


9BT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid applied 


640 


3.278 




1,037 


2.480 


9C 


Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


640 


3.277 




1,037 


2.394 


9CT 


Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


640 


3.278 




1,037 


2.480 













Source; Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.I I. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.3 





Participated in Regular Upward Bound 


Did Not Participate in Regular Upward Bound 


Variable Data Source 


Uncoded 


Sample Size Standard Errorf 


Sample Size Standard Errorf 
















1 5th Follow-Up Survey (Survey) 


Set to Missing Value 


510 5.303 


828 2.214 


2 NSC through 05-06 (NSCT) 


None 


640 4.057 


1,037 1.757 


2T NSC through 03-04 (NSCT) 


None 


640 3.841 


1,037 1.660 


3 Pell Receipt (Student Financial Aid - SFA) 


None 


640 2.805 


1,037 1.074 


4 NSCF /SFA 


None 


640 3.316 


1,037 1.247 


4T NSCT /SFA 


None 


640 3.306 


1,037 1.290 


5A Survey /NSCF /SFA 


Set to 0 


640 3.308 


1,037 1.419 


5AT Survey /NSCT /SFA 


Set to 0 


640 3.281 


1,037 1.458 


5B Survey /NSCF /SFA 


Set to 0 if no aid applied 


640 3.308 


1,036 1.421 


5BT Survey /NSCT /SFA 


Set to 0 if no aid applied 


640 3.281 


1,035 1.448 


5C Survey /NSCF /SFA 


Set to Missing Value 


640 3.308 


1,036 1.421 


5CT Survey /NSCT /SFA 


Set to Missing Value 


638 3.288 


1,034 1.448 


6A Survey / SFA 


Set to 0 


640 3.274 


1,037 1.430 


6B Survey / SFA 


Set to 0 if no aid applied 


617 3.444 


994 1.360 


6C Survey / SFA 


Set to Missing Value 


589 3.516 


953 1.393 


Ik Survey / NSCF 


Set to 0 


640 4.166 


1,037 1.849 


VAT Survey /NSCT 


Set to 0 


640 4.378 


1,037 1.781 


7C Survey /NSCF 


Set to Missing Value 


640 4.166 


1,035 1.855 


7CT Survey / NSCT 


Set to Missing Value 


638 4.380 


1,028 1.801 


8 Survey then NSCF/ SFA 


Set to 0 


640 3.308 


1,037 1.419 


8T Survey then NSCT /SFA 


Set to 0 


640 3.281 


1,037 1.458 


9A Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 


640 3.306 


1,037 1.374 


9AT Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 


640 3.278 


1,037 1.410 


9B Mult Surveys (3rd-5th) / NSCF / SFA 


Set to 0 if no aid applied 


640 3.306 


1,037 1.374 


9BT Mult Surveys (3rd-5th) / NSCT / SFA 


Set to 0 if no aid applied 


640 3.278 


1,037 1.410 


9C Mult Surveys (3rd-5th) / NSCF / SFA 


Set to Missing Value 


640 3.306 


1,037 1.374 


9CT Mult Surveys (3rd-5th) / NSCT / SFA 


Set to Missing Value 


640 3.278 


1,037 1.410 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.I2. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.4 



Fk1iapEtedinR^liarH3wariBound QdNXFtoicipBtemRegia'Lj^aKlBami 



Vaiidie EttaSoroe 


Uioxlad 


Sample Size 


StardaidBrcr)' 


Sample Size 


Standardfiint 


1 


Sih pyiow-Lp Survey (Slirvey) 


Set loMssing Value 


510 


0.000 


828 


1.011 


2 


NBC «ra#iOSB6 (NSCF) 


NotE 


610 


0.024 


1,037 


1.306 


2T 


NBC 4ra#i 0301 (NSd) 


NotE 


610 


0.024 


1,037 


0.562 


3 


ftU Ifeoeift (Stucfent Hnaicial Ad- SFA 


None 


610 


0.000 


1,037 


1.200 


4 


NSCF/ SFA 


None 


610 


0.000 


1,037 


1.^5 


4T 


NSCT/ SFA 


None 


610 


0.000 


1,037 


1.355 


5A 


Sur«y/NSCF/SFA 


SettoO 


610 


0.000 


1,037 


1.247 


5AT 


Sutwy/NSCT/3^A 


SettoO 


610 


0.000 


1,037 


1.291 


SB 


Sutwy/NSCF/SFA 


Set to Oif IB aid applied 


610 


0.000 


1,036 


1.247 


5BT 


Sutwy/NSCT/3^A 


Set to Oif IB aid applied 


610 


0.000 


1,035 


1.293 


SC 


Sutwy/NSCF/SFA 


Set BMsangValiE 


610 


0.000 


1,036 


1.247 


scr 


Sutwy/NSCr/3^A 


Set BMsangValiE 


638 


0.000 


1,031 


1.293 


6A 


Slrvey/SFA 


SettoO 


610 


0.000 


1,037 


1.080 


6B 


Slrvey/SFA 


Set to Oif IB aid apjlied 


617 


0.000 


m 


1.122 


ec 


Sbrvey/SFA 


Set loMssing ValiE 


589 


0.000 


953 


1.160 


7A 


Survey/ NSCF 


&ttoO 


610 


0.000 


1,037 


1.019 


7AT 


Survey/ NSCT 


&ttoO 


610 


0.000 


1,037 


1.088 


7C 


Survey/ NSCF 


Set loMssing Value 


610 


0.000 


1,035 


1.020 


7CT 


Survey/ NSCT 


Set loMssing Value 


638 


0.000 


1,028 


1.093 


8 


Survey then NSCF/ SFA 


&ttoO 


610 


0.000 


1,037 


1.247 


8T 


SurveyttenfSCT/SA 


&ttoO 


610 


0.000 


1,037 


1.291 


9A 


Mlt Sut\e3s (id-Sth) / fNCF/ SFA 


&ttoO 


610 


0.000 


1,037 


1.222 


9AT 


Mlt Sutwjs (3rd-Sth) / NSCT/ SA 


&ttoO 


610 


0.000 


1,037 


1.265 


9B 


Mlt Sut\e3s (id-Sth) / fNCF/ SFA 


Set to Oif IB aid apjlied 


610 


0.000 


1,037 


1.222 


9BT 


Mlt Sut\e3s (id-Sth) / fNCT/ S^A 


Set to Oif no aid q:plied 


610 


0.000 


1,037 


1.265 


9C 


Mlt Sut\e>s (3rd-StH) / NSCF/ SFA 


Set loMssing ValiE 


610 


0.000 


1,037 


1.222 


9CT 


Mlt Sut\e>s (id-Sth) / NSCT/ S^A 


Set loMssing Value 


610 


0.000 


1,037 


1.265 



Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Notes: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.I3. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.5 









Participated in Regular Upward Bound 


Did Not Paiticipate in Regular Upward Bound 


Variables 


Data Source 


Uncoded 


Sanple Size 


Standard Errort 


Sanple Size 


Standard Errort 






















1 


5th Follow-Up Survey (Survey) 


Set to Missing Value 


510 


7.915 


828 


6.696 


2 


NSC through 05-06 (NSCF) 


None 


640 


7.291 


1,037 


7.627 


2T 


NSC through 03-04 (NSCT) 


None 


640 


7.390 


1,037 


7.614 


3 


Pell Receipt ( Student Financial Aid - SFA) 


None 


640 


7.047 


1,037 


5.903 


4 


NSCF / SFA 


None 


640 


7.049 


1,037 


5.660 


4T 


NSCT / SFA 


None 


640 


7.090 


1,037 


5.726 


5A 


Sui-vey/ NSCF /SFA 


Set to 0 


640 


6.851 


1,037 


6.014 


5AT 


Survey /NSCT /SFA 


Set to 0 


640 


6.891 


1,037 


6.076 


5B 


Survey /NSCF /SFA 


Set to 0 if no aid applied 


640 


6.851 


1,036 


5.997 


5BT 


Survey /NSCT /SFA 


Set to 0 if no aid applied 


640 


6.891 


1,035 


6.051 


5C 


Survey /NSCF /SFA 


Set to Missing Value 


640 


6.851 


1,036 


5.997 


5CT 


Survey /NSCT /SFA 


Set to Missing Value 


638 


6.890 


1,034 


6.050 


6A 


Survey / SFA 


Set to 0 


640 


6.884 


1,037 


6.090 


6B 


Survey / SFA 


Set to 0 if no aid applied 


617 


6.960 


994 


6.126 


6C 


Survey / SFA 


Set to Missing Value 


589 


6.893 


953 


6.209 


7A 


Survey / NSCF 


Set to 0 


640 


7.632 


1,037 


6.693 


7AT 


Survey / NSCT 


Set to 0 


640 


7.738 


1,037 


6.665 


7C 


Survey / NSCF 


Set to Missing Value 


640 


7.632 


1,035 


6.692 


7CT 


Survey / NSCT 


Set to Missing Value 


638 


7.742 


1,028 


6.649 


8 


Survey then NSCF /SFA 


Set to 0 


640 


6.851 


1,037 


6.014 


8T 


Survey then NSCT / SFA 


Set to 0 


640 


6.891 


1,037 


6.076 











Source; Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note; NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = 
Student Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.I4. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.6 





Participated in 


Regular Upward Bound 


Did Not Participate 


in Regular Upward Bound 


Postsecondary Credits Earned 


Sample Size 


Standard Error t 


Sample Size 


Standard Errorf 


Two- and four -year college and university 


640 


10.430 


1,037 


9.897 


Two- year colleges 


496 


3.179 


800 


2.608 


Four-year colleges 


496 


8.691 


800 


8.246 


Math and Science 


640 


5.949 


1,037 


4.441 



Source; Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.I5. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.7 



















Participated in Regular Upward Bound 




Did Not Participate in I 


Regular Upward Bound 


Variable 


Outcome / Data Source 


Uncoded 


Sample Size 


Standard Errort 




Sample Size 


Standard Errorf 


































Any Postsecondary Degree 














1 


Survey 


Set to Missing Value 


458 


7.427 




743 


5.516 


2 


NSCF 


None 


640 


7.108 




1,037 


4.741 


2T 


NSCT 


None 


640 


7.109 




1,037 


4.871 


7A 


Survey / NSCF 


Set to 0 


640 


6.683 




1,037 


4.194 


VAT 


Survey / NSCT 


Set to 0 


640 


6.799 




1,037 


4.325 


7B 


Survey / NSCF / SFA 


Set to 0 if no aid applied 


640 


6.683 




1,035 


4.190 


7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


640 


6.799 




1,029 


4.317 


1C 


Survey / NSCF 


Set to Missing Value 


640 


6.683 




1,035 


4.190 


7CT 


Survey / NSCT 


Set to Missing Value 


638 


6.807 




1,028 


4.316 


Highest Degree Completed: Four-Year Degree 












1 


Survey 


Set to Missing Value 


458 


8.859 




743 


7.391 


2 


NSCF 


None 


640 


7.331 




1,037 


5.377 


2T 


NSCT 


None 


640 


7.501 




1,037 


5.501 


7A 


Survey / NSCF 


Set to 0 


640 


7.657 




1,037 


5.212 


VAT 


Survey / NSCT 


Set to 0 


640 


7.825 




1,037 


5.403 


7B 


Survey / NSCF / SFA 


Set to 0 if no aid applied 


640 


7.657 




1,035 


5.216 


7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


640 


7.825 




1,029 


5.404 


1C 


Survey / NSCF 


Set to Missing Value 


640 


7.657 




1,035 


5.216 


7CT 


Survey / NSCT 


Set to Missing Value 


638 


7.827 




1,028 


5.403 


Highest Degree Completed: Two-Year Degree 












1 


Survey 


Set to Missing Value 


458 


3.332 




743 


2.009 


2 


NSCF 


None 


640 


1.637 




1,037 


0.866 


2T 


NSCT 


None 


640 


2.910 




1,037 


0.924 


7A 


Survey / NSCF 


Set to 0 


640 


2.665 




1,037 


1.594 


VAT 


Survey / NSCT 


Set to 0 


640 


3.291 




1,037 


1.708 


7B 


Survey / NSCF / SFA 


Set to 0 if no aid applied 


640 


2.665 




1,035 


1.596 


7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


640 


3.291 




1,029 


1.715 


1C 


Survey / NSCF 


Set to Missing Value 


640 


2.665 




1,035 


1.596 


7CT 


Survey / NSCT 


Set to Missing Value 


638 


3.290 




1,028 


1.715 


Highest Degree Completed: Other Degree 












1 


Survey 


Set to Missing Value 


458 


2.075 




743 


1.776 


2 


NSCF 


None 


640 


0.011 




1,037 


0.771 


2T 


NSCT 


None 


640 


0.011 




1,037 


0.558 


7A 


Survey / NSCF 


Set to 0 


640 


1.561 




1,037 


1.373 


VAT 


Survey / NSCT 


Set to 0 


640 


1.566 




1,037 


1.351 


7B 


Survey / NSCF / SFA 


Set to 0 if no aid applied 


640 


1.561 




1,035 


1.376 


7BT 


Survey / NSCT / SFA 


Set to 0 if no aid applied 


640 


1.566 




1,029 


1.362 


1C 


Survey / NSCF 


Set to Missing Value 


640 


1.561 




1,035 


1.376 


7CT 


Survey / NSCT 


Set to Missing Value 


638 


1.565 




1,028 


1.362 













Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

Note: NSCF = National Student Clearinghouse full; NSCT = National Student Clearinghouse truncated; SFA = Student 
Financial Aid. 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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EXHIBIT C.I6. SAMPLE SIZES AND STANDARD ERRORS EOR REPORTED IMPACT ESTIMATES IN 

APPENDIX B, EXHIBIT B.8 







Participated in Regular Upward Bound 


Did Not Participate in Regular Upward Bound 


Field of Study 


Sample Size 


Standard Errorj 


Sample Size 


Standard Errorf 














Field of Study at Most Recent Postsecondary Institution 










All Postsecondary Institutions 










Math and science fields, excluding social sciences 


486 


7.321 


785 


5.900 


Social science fields 


486 


6.134 


785 


3.743 


Other fields 


486 


6.823 


785 


6.005 


Four-year colleges and universities 










Math and science fields, excluding social sciences 


486 


7.019 


785 


5.474 


Social science fields 


486 


5.931 


785 


4.015 


Other fields 


486 


6.388 


785 


5.944 


Earned Degree or Certificate in Field 










All Postsecondary Institutions 










Math and science fields, excluding social sciences 


486 


6.111 


785 


4.418 


Social science fields 


486 


4.352 


785 


2.994 


Other fields 


486 


5.606 


785 


4.438 


Four-year colleges and universities 










Math and science fields, excluding social sciences 


486 


5.629 


785 


5.036 


Social science fields 


486 


3.932 


785 


3.399 


Other fields 


486 


4.943 


785 


4.485 











Source: Mathematica Policy Research, Inc., analysis file “ITT_UBMS.sas.” 

t Standard errors account for project clustering and were estimated using Taylor series linearization methods. 
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